-- Logs begin at Wed 2022-09-14 00:20:42 UTC, end at Wed 2022-09-14 01:31:31 UTC. -- Sep 14 00:50:12.046712 dv-cinder-857528-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 14 00:50:12.056179 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 14 00:50:12.058182 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Sep 14 00:50:12 2022] *** Sep 14 00:50:12.058182 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 14 00:50:12.058182 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 14 00:50:12.058182 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: nodename: dv-cinder-857528-1-iscsi Sep 14 00:50:12.058182 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: machine: x86_64 Sep 14 00:50:12.058182 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: clock source: unix Sep 14 00:50:12.058182 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: pcre jit disabled Sep 14 00:50:12.058182 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: detected number of CPU cores: 4 Sep 14 00:50:12.058182 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: current working directory: / Sep 14 00:50:12.058182 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: detected binary path: /usr/bin/uwsgi-core Sep 14 00:50:12.058182 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: your processes number limit is 47680 Sep 14 00:50:12.058182 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: your memory page size is 4096 bytes Sep 14 00:50:12.059568 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: detected max file descriptor number: 2048 Sep 14 00:50:12.059568 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: lock engine: pthread robust mutexes Sep 14 00:50:12.059568 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: thunder lock: enabled Sep 14 00:50:12.059568 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 14 00:50:12.059568 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 14 00:50:12.098529 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: Python main interpreter initialized at 0x55e86f5f99c0 Sep 14 00:50:12.098529 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: python threads support enabled Sep 14 00:50:12.098529 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: your server socket listen backlog is limited to 100 connections Sep 14 00:50:12.098529 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: your mercy for graceful operations on workers is 90 seconds Sep 14 00:50:12.098885 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: mapped 403077 bytes (393 KB) for 2 cores Sep 14 00:50:12.098965 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: *** Operational MODE: preforking *** Sep 14 00:50:12.099211 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: *** uWSGI is running in multiple interpreter mode *** Sep 14 00:50:12.099211 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: spawned uWSGI master process (pid: 81405) Sep 14 00:50:12.099406 dv-cinder-857528-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 14 00:50:12.101855 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: spawned uWSGI worker 1 (pid: 81417, cores: 1) Sep 14 00:50:12.101855 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: spawned uWSGI worker 2 (pid: 81418, cores: 1) Sep 14 00:50:12.101855 dv-cinder-857528-1-iscsi devstack@keystone.service[81405]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 14 00:50:16.474593 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 14 00:50:16.474593 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: from cryptography.utils import int_from_bytes Sep 14 00:50:16.475385 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 14 00:50:16.475385 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: from cryptography.utils import int_from_bytes Sep 14 00:50:16.475507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 14 00:50:16.475507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: from cryptography.utils import int_from_bytes Sep 14 00:50:16.475947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 14 00:50:16.475947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: from cryptography.utils import int_from_bytes Sep 14 00:50:16.525819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 14 00:50:16.525998 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 14 00:50:16.526190 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] command line args: [] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 14 00:50:16.526461 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 14 00:50:16.526548 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 14 00:50:16.526628 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ================================================================================ {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 14 00:50:16.526710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 14 00:50:16.526855 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] command line args: [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 14 00:50:16.526976 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] admin_token = **** {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.527112 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] config_dir = [] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.527193 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 14 00:50:16.527275 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ================================================================================ {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 14 00:50:16.527347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.527471 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] config_source = [] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.527547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] admin_token = **** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.527685 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] debug = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.527802 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] config_dir = [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.527895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.528145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.528145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] config_source = [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.528263 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.528326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] debug = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.528476 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.528560 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.528680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.528758 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.528911 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.529001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.529140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] list_limit = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.529225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.529326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] log_config_append = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.529400 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.529498 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.529617 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] list_limit = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.529695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] log_dir = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.529799 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] log_config_append = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.529873 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] log_file = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.529982 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.530064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.530181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] log_dir = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.530294 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.530380 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] log_file = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.530520 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.530625 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.530747 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.530858 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.530932 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.531051 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.531151 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.531302 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.531406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.531486 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.531574 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.531657 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.531789 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.531937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.532059 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.532171 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.532315 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.532421 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.532562 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.532671 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.532784 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.532925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.533032 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.533193 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.533301 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.533407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.533553 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.533659 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.533772 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.534572 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] publish_errors = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.534695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.534801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.534946 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.535052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] publish_errors = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.535192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.535307 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.535420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.535572 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.535689 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.535849 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.535990 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] standard_threads = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.536102 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.536208 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.536318 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.536435 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.536435 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.536609 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] standard_threads = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.536681 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] use_journal = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.536791 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.536870 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] use_json = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.536870 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] use_stderr = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.537075 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.537182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] use_syslog = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.537255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.537255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] use_journal = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.537393 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.537472 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] use_json = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.537550 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.537621 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] use_stderr = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.537621 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] use_syslog = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.537752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.537852 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 00:50:16.537941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.538078 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.538162 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.538322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.538400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.538559 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.538646 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.538808 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.538885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.539044 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.539121 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.539285 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.539371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.539528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.539677 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.539793 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.539914 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.540028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.540150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.540269 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.540381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.540493 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.540604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.540721 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.540842 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.540961 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.541074 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.541186 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.541297 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.541414 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.541535 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.541647 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.541758 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.541899 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.542043 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.542161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.542348 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.542470 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.542582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.542694 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.542811 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.542933 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.543047 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.543183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.543298 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.543409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.543520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.543694 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.543810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.543921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.544034 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.544156 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.544270 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.544393 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.544506 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.544617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.544764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.544880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.544998 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.545120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.545239 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.545352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.545464 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.545574 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.545692 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.545812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.545931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.546049 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.546162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.546274 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.546385 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.546504 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.546617 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.546728 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.546839 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.546950 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.547061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.547196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.547316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.547429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.547429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.547653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.547765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.547958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.547958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.548171 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.548282 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.548392 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.548502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.548620 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.548620 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.548829 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.548945 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.549056 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.549172 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.549293 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.549411 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.549566 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.549679 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.549789 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.549900 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.550007 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.550109 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.550205 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.550300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.550393 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.550487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.550589 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.550684 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.550777 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.550916 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.551027 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.551132 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.551248 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.551353 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.551464 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.551617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.551742 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.551864 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.551978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.552097 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.552202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.552313 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.552313 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.552539 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.552652 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.552775 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.552887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.552999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.553122 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.553236 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.553348 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.553467 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.553467 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.553669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.553779 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.553891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.553891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.554110 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.554226 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.554338 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.554453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.554573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.554686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.554805 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.554925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.555029 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.555140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.555251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.555362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.555481 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.555607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.555721 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.555832 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.555944 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.556054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.556177 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.556292 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.556404 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.556515 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.556634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.556747 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.556866 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.556979 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.557091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.557203 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.557315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.557426 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.557545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.557664 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.557777 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.557889 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.558000 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.558111 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.558231 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.558358 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.558585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.558718 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.558834 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.558946 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.559129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.559246 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.559365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.559479 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.559605 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.559734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.559857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.559980 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.560094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.560207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.560207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.560425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.560425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.560652 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.560768 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.560880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.560992 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.561110 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.561224 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.561344 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.561458 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.561571 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.561683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.561796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.561915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.562036 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.562160 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.562274 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] auth.external = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.562394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.562519 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.562639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.562783 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.562908 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.563031 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.563031 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] auth.password = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.563245 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.563359 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] auth.token = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.563480 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.563628 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.563754 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] auth.external = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.563870 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.563984 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.564101 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.564215 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.564329 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.564450 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.564565 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.564678 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] auth.password = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.564792 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.564905 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] auth.token = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.565019 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.565139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.565254 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] credential.caching = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.565373 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.565487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.565599 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.565712 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.565868 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.566037 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.566158 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.566272 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.566386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.566509 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.566632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.566748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.566868 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] credential.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.566984 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.567099 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.567212 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.567212 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.567436 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.567566 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.567684 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.567684 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.567896 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.568022 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.568140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.568253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.568373 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.568488 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.568602 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.568723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.568839 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.568966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.569155 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] federation.caching = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.569155 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.569402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.569525 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.569640 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.569753 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.569867 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.569980 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.570093 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.570220 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.570343 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.570458 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.570571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.570684 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.570806 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] federation.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.570927 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.571041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.571161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.571277 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.571390 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.571519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.571681 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.571806 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.571926 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.caching = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.572040 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.572154 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.572275 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.572390 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.572554 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.572673 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.572797 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.572911 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.573100 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.573218 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.573339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.573454 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.573568 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.573715 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.573840 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.573956 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.574116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.574233 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.574346 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.574460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.574460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.574950 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.575066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.575179 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.575293 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.575406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.575530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.575661 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.575774 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.575888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.576001 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.576114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.576240 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.576357 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.576357 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.576561 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.576681 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.576796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.576917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.577032 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.577145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.577259 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.577372 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.577485 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.577608 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.577723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.577837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.577950 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.577950 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.578163 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.578285 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.578401 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.578514 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.578629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.578743 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.578856 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.578978 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.579093 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.579206 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.579319 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.579433 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.579546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.579689 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.579805 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.579919 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.580033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.580146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.580259 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.580381 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.580497 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.580610 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.581256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.581379 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.581502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.581618 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.581733 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.581847 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.581959 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.582073 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.582194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.582309 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.582423 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.582536 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.582649 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.582649 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.582649 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.582984 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.583098 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.583211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.583324 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.583437 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.583549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.583736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.583851 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.583966 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.584080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.584193 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.584319 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.584435 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.584549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.584668 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.584781 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.584897 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.585019 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.585134 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.585248 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.585361 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.585361 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.585574 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.585696 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.585812 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.585926 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.586040 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.586152 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.586266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.586388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.586503 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.586617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.586731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.586844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.586958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.587080 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.587195 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.587308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.587422 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.587535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.587669 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.587795 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.587940 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.588057 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.588170 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.588282 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.588394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.588514 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.588661 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.588778 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.588892 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.589005 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.589119 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.589240 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.589356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.589469 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.589582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.589695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.589808 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.589930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.590045 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.590160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.590274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.590387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.590501 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.590623 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.590739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.590853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.590966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.591079 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.591194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.591316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.591432 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.591545 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.591692 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.591806 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.591919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.592044 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.592160 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.592275 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.592388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.592501 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.592614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.592735 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.592735 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.592949 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.593062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.593175 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.593295 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] resource.caching = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.593418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.593533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.593653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.593767 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.593881 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.593994 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.594115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.594231 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.594346 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.594459 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.594576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.594690 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.594811 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.594926 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.595040 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.595153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.595266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] resource.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.595380 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.595502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.595632 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] role.caching = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.595747 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.595861 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] role.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.595975 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.596163 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.596281 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.597013 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.597133 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.597248 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.597516 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.597652 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.597769 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.597885 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.598000 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.598114 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.598228 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.598351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.598467 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] role.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.598583 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.598698 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] role.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.598818 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.598933 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.599056 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.599172 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.599286 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.599399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.599514 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.599686 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.599818 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.599936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.600051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.600165 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.600165 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.600379 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.600379 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.600604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.600718 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.600832 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.600832 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.601046 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.601167 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.601282 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.601282 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.601495 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.601608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.601723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.601846 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.601961 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.602076 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.602190 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.602304 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.602425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.602530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.602643 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.602758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.602758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.602971 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.603093 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.603208 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.603322 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.603435 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.603548 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.603682 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.603811 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.603928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.604042 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.604155 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.604268 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.604381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.604502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] token.caching = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.604617 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.604730 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.604844 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.604958 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.605071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.605193 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.605311 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.605431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.605546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.605659 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.605772 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.605894 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.606009 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.606123 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.606237 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.606349 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.606462 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.606583 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.606698 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] token.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.606811 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.606925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.607038 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.607152 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.607273 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.607388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.607501 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.607636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.607750 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.607864 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.607990 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.608115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.608238 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.608353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.608466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.608579 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.608692 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.connection = **** {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.608813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.608928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.609042 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.609155 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.609269 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.609382 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.609503 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.609618 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.609731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.609844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.609957 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.610071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.610192 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.610316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.610442 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.610558 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.610671 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.610785 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.610785 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.610998 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.connection = **** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.611122 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.611237 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.611352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.611466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.611595 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.611711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.611838 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.611977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.612096 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.612285 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.612473 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.612606 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.612790 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.612915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.613030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.613145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.613261 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81418) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 14 00:50:16.613375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.613375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.613375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.613375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.613375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.613375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.613375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.613375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.614206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.614206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 00:50:16.614206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81417) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 14 00:50:16.640687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.640917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.641142 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.642256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.642494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.642726 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.649913 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.651695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.678042 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.678267 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.680398 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.680638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.732071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.732420 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.780893 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.781122 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.781605 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.781845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.937141 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.937369 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.937590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.937678 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.937808 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.937895 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.938022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.938107 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.938235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.938329 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.938527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.938745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.961818 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.962055 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.962221 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.962327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.962439 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.962524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.962666 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.962750 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.962880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.962964 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.963091 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.963178 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.963308 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.963526 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.974683 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.975148 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.987902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.988087 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.988315 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.988534 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.988620 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.988745 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.988827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.988955 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.989046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.989164 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81418) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.989246 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.989393 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.989611 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:16.989828 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81417) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 00:50:17.005661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.005872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 14 00:50:17.006213 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.006295 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.006414 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 14 00:50:17.006511 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 14 00:50:17.006690 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.006773 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.006955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 14 00:50:17.007037 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.007215 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.007340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.007471 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.007576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.007771 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.008035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.017274 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.017551 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.017822 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.017904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.018107 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.018205 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.018506 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.018821 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.025082 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.025939 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.027977 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.028263 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.028547 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.028832 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.028914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.029115 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.029212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.029379 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.029463 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.029650 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.029731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.029924 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.030006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.030195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.030276 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.030468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.030557 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.030735 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.030816 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.031010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.031293 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.031592 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.054459 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.055418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.056005 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.056292 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.056925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.057218 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.060931 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.061214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.061496 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.061904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.062194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.062483 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.067863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.068149 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.068895 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.069188 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.073721 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.074013 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.074295 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.074579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.074855 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.074950 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.075130 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.075213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.075399 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.075481 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.075690 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.075776 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.075999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.076279 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.076556 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.076830 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.091959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.092237 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.092921 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.093204 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.096106 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.097176 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.099719 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.100016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.100444 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.100746 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.104212 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.104987 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.107355 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.108237 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.110573 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.110872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.111159 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.111446 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.111526 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.111752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.111848 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.112576 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.112681 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.112992 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.113315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.113629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.128370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.128659 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.128928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.129195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.129682 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.129976 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.130249 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.130518 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.136006 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.137434 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.137890 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.139215 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.139309 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.139531 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.140790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.141072 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.142240 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.142524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.142807 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.143097 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.144181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.144383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.144671 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.144957 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.145250 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.145659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.155245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.155535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.155844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.156130 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.156512 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.156796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.156880 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.157087 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.157169 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.157405 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.157694 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.158080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.158415 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.158657 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.172643 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.174100 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.176268 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.177639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.179219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.180582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.180711 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.182129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.182231 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.182448 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.182839 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.183622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.183914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.184318 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.189196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.191318 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.191417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.192894 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.194195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.194596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.194886 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.195166 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.195827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.196239 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.196521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.196801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.203046 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.203339 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.203644 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.204662 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.204963 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.205255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.210636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81418) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.210922 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.211201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.211480 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.211783 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.212058 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.212249 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81417) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 00:50:17.212344 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.212541 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.212623 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.212826 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.212907 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.213106 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.213197 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.213382 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.213465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.213654 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.213736 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.213923 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.214004 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.214193 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81418) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.214274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.214469 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.214751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.215035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.215421 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.215771 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.216048 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81417) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 00:50:17.249722 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e86f5f99c0 pid: 81418 (default app) Sep 14 00:50:17.250474 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e86f5f99c0 pid: 81417 (default app) Sep 14 00:50:17.253604 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cebe1-29c8-4622-a2b8-2e36559239e5 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:17.253888 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cebe1-29c8-4622-a2b8-2e36559239e5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:17.254162 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cebe1-29c8-4622-a2b8-2e36559239e5 None None] PATH_INFO: `/v3` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:17.258293 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Wed Sep 14 00:50:17 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:24.134059 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645f7f2-a230-41f8-8de7-1b2f2d487d18 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:24.134576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645f7f2-a230-41f8-8de7-1b2f2d487d18 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:24.134681 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645f7f2-a230-41f8-8de7-1b2f2d487d18 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:24.138871 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:50:24.145238 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231307e-29d4-4ef9-9bd3-8aa4b57a522c None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:24.145711 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231307e-29d4-4ef9-9bd3-8aa4b57a522c None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:24.145803 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231307e-29d4-4ef9-9bd3-8aa4b57a522c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:24.216550 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: INFO dbcounter [None req-f231307e-29d4-4ef9-9bd3-8aa4b57a522c None None] Registered counter for database keystone Sep 14 00:50:24.235497 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG oslo_db.sqlalchemy.engines [None req-f231307e-29d4-4ef9-9bd3-8aa4b57a522c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81418) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 14 00:50:24.242065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writer thread running {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 14 00:50:24.422060 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG passlib.handlers.bcrypt [None req-f231307e-29d4-4ef9-9bd3-8aa4b57a522c None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81418) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 14 00:50:24.422465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG passlib.handlers.bcrypt [None req-f231307e-29d4-4ef9-9bd3-8aa4b57a522c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81418) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 14 00:50:24.481399 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-f231307e-29d4-4ef9-9bd3-8aa4b57a522c None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:24.483027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f231307e-29d4-4ef9-9bd3-8aa4b57a522c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:24.616576 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 14 00:50:24.911327 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645f7f2-a230-41f8-8de7-1b2f2d487d18 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:24.980957 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: INFO dbcounter [None req-f645f7f2-a230-41f8-8de7-1b2f2d487d18 None None] Registered counter for database keystone Sep 14 00:50:25.002687 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG oslo_db.sqlalchemy.engines [None req-f645f7f2-a230-41f8-8de7-1b2f2d487d18 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81417) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 14 00:50:25.009151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writer thread running {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 14 00:50:25.130277 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645f7f2-a230-41f8-8de7-1b2f2d487d18 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:25.134489 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834da5d0-ad99-453e-ab6f-07bc49fe9d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:25.135118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834da5d0-ad99-453e-ab6f-07bc49fe9d32 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:25.135403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834da5d0-ad99-453e-ab6f-07bc49fe9d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:25.135709 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834da5d0-ad99-453e-ab6f-07bc49fe9d32 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:25.144440 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834da5d0-ad99-453e-ab6f-07bc49fe9d32 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:25.196793 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: WARNING keystone.common.rbac_enforcer.enforcer [None req-834da5d0-ad99-453e-ab6f-07bc49fe9d32 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 14 00:50:25.200984 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834da5d0-ad99-453e-ab6f-07bc49fe9d32 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:25.206879 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1324 bytes} [Wed Sep 14 00:50:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 00:50:25.212803 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231307e-29d4-4ef9-9bd3-8aa4b57a522c None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:25.230155 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231307e-29d4-4ef9-9bd3-8aa4b57a522c None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:25.234542 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a22f63-387e-45f6-a6cf-143b8f36cab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:25.235693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a22f63-387e-45f6-a6cf-143b8f36cab3 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:25.235977 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a22f63-387e-45f6-a6cf-143b8f36cab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:25.236249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a22f63-387e-45f6-a6cf-143b8f36cab3 admin admin] PATH_INFO: `/v3/projects/724534edd35a41c585d8ed624130e96a` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:25.238349 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a22f63-387e-45f6-a6cf-143b8f36cab3 admin admin] RBAC: Authorizing `identity:get_project(project_id=724534edd35a41c585d8ed624130e96a)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:25.290794 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c1a22f63-387e-45f6-a6cf-143b8f36cab3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 14 00:50:25.291131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a22f63-387e-45f6-a6cf-143b8f36cab3 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:25.295646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 3/5] 10.222.0.42 () {56 vars in 1207 bytes} [Wed Sep 14 00:50:25 2022] GET /identity/v3/projects/724534edd35a41c585d8ed624130e96a => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:27.443079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08339cad-5b03-4657-8d3a-83cf0c426f72 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:27.443627 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08339cad-5b03-4657-8d3a-83cf0c426f72 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:27.443712 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08339cad-5b03-4657-8d3a-83cf0c426f72 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:27.447808 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 3/6] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:50:27.453179 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c8243-f4ea-496c-bd5d-bd44e2494493 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:27.453543 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c8243-f4ea-496c-bd5d-bd44e2494493 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:27.453745 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c8243-f4ea-496c-bd5d-bd44e2494493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:27.474590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-5e4c8243-f4ea-496c-bd5d-bd44e2494493 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:27.476155 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5e4c8243-f4ea-496c-bd5d-bd44e2494493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:27.484136 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 14 00:50:27.755178 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08339cad-5b03-4657-8d3a-83cf0c426f72 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:27.767408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08339cad-5b03-4657-8d3a-83cf0c426f72 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:27.772653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc1195-53a1-4072-b314-792d86608620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:27.773256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc1195-53a1-4072-b314-792d86608620 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:27.773684 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc1195-53a1-4072-b314-792d86608620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:27.773965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc1195-53a1-4072-b314-792d86608620 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:27.781579 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc1195-53a1-4072-b314-792d86608620 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:27.783344 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc1195-53a1-4072-b314-792d86608620 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:27.792563 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1324 bytes} [Wed Sep 14 00:50:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 14 00:50:27.797946 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c8243-f4ea-496c-bd5d-bd44e2494493 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:27.807377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c8243-f4ea-496c-bd5d-bd44e2494493 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:27.811752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d8b0b-5956-43c0-89ab-8c06cb6eff9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:27.812385 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d8b0b-5956-43c0-89ab-8c06cb6eff9c admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:27.812672 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d8b0b-5956-43c0-89ab-8c06cb6eff9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:27.812951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d8b0b-5956-43c0-89ab-8c06cb6eff9c admin admin] PATH_INFO: `/v3/users/9dea2911925e405b852c2fe0f9bee81d` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:27.814265 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d8b0b-5956-43c0-89ab-8c06cb6eff9c admin admin] RBAC: Authorizing `identity:get_user(user_id=9dea2911925e405b852c2fe0f9bee81d)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:27.816284 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d8b0b-5956-43c0-89ab-8c06cb6eff9c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:27.817812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 5/9] 10.222.0.42 () {56 vars in 1198 bytes} [Wed Sep 14 00:50:27 2022] GET /identity/v3/users/9dea2911925e405b852c2fe0f9bee81d => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:30.029419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33d992-a121-4113-b3fb-f8ae6838022a None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:30.029961 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33d992-a121-4113-b3fb-f8ae6838022a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:30.029961 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33d992-a121-4113-b3fb-f8ae6838022a None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:30.031308 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:50:30.037207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc05e7-8473-430a-ad05-340cd2c679c0 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:30.037607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc05e7-8473-430a-ad05-340cd2c679c0 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:30.037764 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc05e7-8473-430a-ad05-340cd2c679c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:30.055706 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-64bc05e7-8473-430a-ad05-340cd2c679c0 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:30.056878 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-64bc05e7-8473-430a-ad05-340cd2c679c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:30.075890 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451a7aa-c221-495e-b881-e4b73101b4d7 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:30.076181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451a7aa-c221-495e-b881-e4b73101b4d7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:30.076458 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451a7aa-c221-495e-b881-e4b73101b4d7 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:30.077787 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 6/11] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:50:30.083572 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a13ed3-2508-4e2f-94f3-f1370ee9455d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:30.083868 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a13ed3-2508-4e2f-94f3-f1370ee9455d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:30.084149 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a13ed3-2508-4e2f-94f3-f1370ee9455d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:30.127238 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG passlib.handlers.bcrypt [None req-f3a13ed3-2508-4e2f-94f3-f1370ee9455d None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81417) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 14 00:50:30.127770 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG passlib.handlers.bcrypt [None req-f3a13ed3-2508-4e2f-94f3-f1370ee9455d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81417) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 14 00:50:30.139341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 6/12] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:50:30.177570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f3a13ed3-2508-4e2f-94f3-f1370ee9455d None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:30.179158 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f3a13ed3-2508-4e2f-94f3-f1370ee9455d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:30.279234 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 7/13] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:50:30.410916 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc05e7-8473-430a-ad05-340cd2c679c0 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:30.435703 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:30.515872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc05e7-8473-430a-ad05-340cd2c679c0 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:30.548870 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89439aa8-d559-4904-a085-e5da88ebb3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:30.550149 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89439aa8-d559-4904-a085-e5da88ebb3c6 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:30.550429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89439aa8-d559-4904-a085-e5da88ebb3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:30.550706 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89439aa8-d559-4904-a085-e5da88ebb3c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:30.550999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a13ed3-2508-4e2f-94f3-f1370ee9455d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:30.562613 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89439aa8-d559-4904-a085-e5da88ebb3c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:50:30.571700 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-89439aa8-d559-4904-a085-e5da88ebb3c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:50:30.571700 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:50:30.571700 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:50:30.571700 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:50:30.571700 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:50:30.571700 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:50:30.571700 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:50:30.571700 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:50:30.571700 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:50:30.571700 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:50:30.571700 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:50:30.571700 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:50:30.571700 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:50:30.571700 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:50:30.571700 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:50:30.571700 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:50:30.572863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:50:30.572863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:50:30.572863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:50:30.572863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:50:30.572863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:50:30.572863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:50:30.572863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:50:30.572863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:50:30.572863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:50:30.572863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:50:30.572863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:50:30.572863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:50:30.572863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:50:30.572863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:50:30.572863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:50:30.572863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:50:30.574016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:50:30.574016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:50:30.574016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:50:30.574016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:50:30.574016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:50:30.574016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:50:30.574016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:50:30.574016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:50:30.574016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:50:30.574016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:50:30.574016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:50:30.574016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:50:30.574016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 7/14] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:50:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:50:30.578518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89439aa8-d559-4904-a085-e5da88ebb3c6 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:30.655732 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a13ed3-2508-4e2f-94f3-f1370ee9455d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:30.657988 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:30.661687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:30.680710 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89439aa8-d559-4904-a085-e5da88ebb3c6 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:30.694910 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bb072-34c4-4dd8-be2a-5d23cc86f421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:30.695823 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bb072-34c4-4dd8-be2a-5d23cc86f421 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:30.696113 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bb072-34c4-4dd8-be2a-5d23cc86f421 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:30.696384 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bb072-34c4-4dd8-be2a-5d23cc86f421 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:30.716580 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c6e80-2436-4d88-8730-2d581b46cc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:30.717194 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c6e80-2436-4d88-8730-2d581b46cc35 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:30.717478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c6e80-2436-4d88-8730-2d581b46cc35 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:30.717752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c6e80-2436-4d88-8730-2d581b46cc35 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:30.718423 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c6e80-2436-4d88-8730-2d581b46cc35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:50:30.718730 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c6e80-2436-4d88-8730-2d581b46cc35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:30.720287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c6e80-2436-4d88-8730-2d581b46cc35 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:30.732021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 8/15] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:50:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:50:30.738284 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c6e80-2436-4d88-8730-2d581b46cc35 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:30.766205 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330bb072-34c4-4dd8-be2a-5d23cc86f421 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:30.767763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330bb072-34c4-4dd8-be2a-5d23cc86f421 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:30.838099 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c6e80-2436-4d88-8730-2d581b46cc35 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:30.867386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 8/16] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:50:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 14 00:50:30.872607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c27807-1ead-4c25-a4b2-1255a3837cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:30.872998 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bb072-34c4-4dd8-be2a-5d23cc86f421 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:30.873247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c27807-1ead-4c25-a4b2-1255a3837cef None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:30.873531 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c27807-1ead-4c25-a4b2-1255a3837cef None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:30.873808 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c27807-1ead-4c25-a4b2-1255a3837cef None admin] PATH_INFO: `/v3/users/9dea2911925e405b852c2fe0f9bee81d` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:30.874955 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c27807-1ead-4c25-a4b2-1255a3837cef None admin] RBAC: Authorizing `identity:get_user(user_id=9dea2911925e405b852c2fe0f9bee81d)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:30.875659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:30.876462 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c27807-1ead-4c25-a4b2-1255a3837cef None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:30.878930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 9/17] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:50:30 2022] GET /identity/v3/users/9dea2911925e405b852c2fe0f9bee81d => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:50:30.884229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c27807-1ead-4c25-a4b2-1255a3837cef None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:30.899258 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:30.975345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bb072-34c4-4dd8-be2a-5d23cc86f421 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:30.982096 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c27807-1ead-4c25-a4b2-1255a3837cef None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:31.008426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c4d6b-11f9-4a4d-a559-bc635b5b3c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:31.009292 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c4d6b-11f9-4a4d-a559-bc635b5b3c29 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:31.009578 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c4d6b-11f9-4a4d-a559-bc635b5b3c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:31.009863 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c4d6b-11f9-4a4d-a559-bc635b5b3c29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:31.016946 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9213c83-f912-4f18-bf9c-e046311c3b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:31.017570 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9213c83-f912-4f18-bf9c-e046311c3b5a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:31.017852 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9213c83-f912-4f18-bf9c-e046311c3b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:31.018128 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9213c83-f912-4f18-bf9c-e046311c3b5a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:31.019282 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9213c83-f912-4f18-bf9c-e046311c3b5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:31.020763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9213c83-f912-4f18-bf9c-e046311c3b5a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:31.022287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 10/18] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:50:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:50:31.028042 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9213c83-f912-4f18-bf9c-e046311c3b5a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:31.028656 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c4d6b-11f9-4a4d-a559-bc635b5b3c29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:31.030249 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c4d6b-11f9-4a4d-a559-bc635b5b3c29 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:31.048904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-b92c4d6b-11f9-4a4d-a559-bc635b5b3c29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:50:31.048904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:50:31.048904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:50:31.048904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:50:31.048904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:50:31.048904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:50:31.048904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:50:31.048904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:50:31.048904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:50:31.048904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:50:31.048904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:50:31.048904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:50:31.048904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:50:31.048904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:50:31.048904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:50:31.048904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:50:31.049955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:50:31.049955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:50:31.049955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:50:31.049955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:50:31.049955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:50:31.049955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:50:31.049955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:50:31.049955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:50:31.049955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:50:31.049955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:50:31.049955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:50:31.049955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:50:31.049955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:50:31.049955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:50:31.049955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:50:31.049955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:50:31.049955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:50:31.050752 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:50:31.050752 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:50:31.050752 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:50:31.050752 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 9/19] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:50:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:50:31.055972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c4d6b-11f9-4a4d-a559-bc635b5b3c29 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:31.110162 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:31.111235 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:31.130326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9213c83-f912-4f18-bf9c-e046311c3b5a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:31.158513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c4d6b-11f9-4a4d-a559-bc635b5b3c29 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:31.164041 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc2acb-eecf-4ffb-9ec2-65f566e0d31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:31.164649 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc2acb-eecf-4ffb-9ec2-65f566e0d31a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:31.164927 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc2acb-eecf-4ffb-9ec2-65f566e0d31a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:31.165200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc2acb-eecf-4ffb-9ec2-65f566e0d31a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:31.165919 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc2acb-eecf-4ffb-9ec2-65f566e0d31a None admin] RBAC: Adding query filter params (role.id=ada268ea63354a0297070369dfe3c6cc, scope.domain.id=default, user.id=9dea2911925e405b852c2fe0f9bee81d) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:50:31.166222 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc2acb-eecf-4ffb-9ec2-65f566e0d31a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:31.167690 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc2acb-eecf-4ffb-9ec2-65f566e0d31a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:31.174074 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 11/20] 10.222.0.42 () {56 vars in 1322 bytes} [Wed Sep 14 00:50:31 2022] GET /identity/v3/role_assignments?user.id=9dea2911925e405b852c2fe0f9bee81d&scope.domain.id=default&role.id=ada268ea63354a0297070369dfe3c6cc => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:50:31.193311 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c1573-9969-4497-90cc-985b4657e84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:31.193949 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c1573-9969-4497-90cc-985b4657e84f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:31.194229 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c1573-9969-4497-90cc-985b4657e84f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:31.194509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c1573-9969-4497-90cc-985b4657e84f None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:31.195173 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c1573-9969-4497-90cc-985b4657e84f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:50:31.195475 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c1573-9969-4497-90cc-985b4657e84f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:31.196962 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c1573-9969-4497-90cc-985b4657e84f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:31.217568 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 10/21] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:50:33.663922 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41753e05-1dbc-4a61-a440-ff98feda3080 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:33.664409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41753e05-1dbc-4a61-a440-ff98feda3080 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:33.664512 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41753e05-1dbc-4a61-a440-ff98feda3080 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:33.665861 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 12/22] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:50:33.671706 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060a3f3-e52d-4b7d-92a9-68961cb089bc None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:33.672151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060a3f3-e52d-4b7d-92a9-68961cb089bc None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:33.672291 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060a3f3-e52d-4b7d-92a9-68961cb089bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:33.699040 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2060a3f3-e52d-4b7d-92a9-68961cb089bc None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:33.700257 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2060a3f3-e52d-4b7d-92a9-68961cb089bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:33.801672 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 11/23] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:50:34.041693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82054ff-bb2e-4816-841d-b2bf20100fa3 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:34.042379 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82054ff-bb2e-4816-841d-b2bf20100fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:34.042876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82054ff-bb2e-4816-841d-b2bf20100fa3 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:34.044433 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 13/24] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:50:34.049721 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe4437-137a-4771-949c-a93eee14f64c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:34.050290 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe4437-137a-4771-949c-a93eee14f64c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:34.050759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe4437-137a-4771-949c-a93eee14f64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:34.076640 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-fcbe4437-137a-4771-949c-a93eee14f64c None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:34.078281 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fcbe4437-137a-4771-949c-a93eee14f64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:34.137739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:34.170463 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 12/25] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:50:34.287590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24de9d2-6515-420a-8e51-2df4bebb9560 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:34.288231 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24de9d2-6515-420a-8e51-2df4bebb9560 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:34.288715 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24de9d2-6515-420a-8e51-2df4bebb9560 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:34.290498 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 14/26] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:50:34.301616 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b68e9c-6c5c-4de9-b82f-41c97e096791 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:34.301909 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b68e9c-6c5c-4de9-b82f-41c97e096791 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:34.302193 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b68e9c-6c5c-4de9-b82f-41c97e096791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:34.311374 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24de9d2-6515-420a-8e51-2df4bebb9560 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:34.328646 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f7b68e9c-6c5c-4de9-b82f-41c97e096791 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:34.330212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f7b68e9c-6c5c-4de9-b82f-41c97e096791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:34.461409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 13/27] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:50:34.462773 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2d383-b99a-4d48-bbd9-40d61aac261b None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:34.463059 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2d383-b99a-4d48-bbd9-40d61aac261b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:34.463333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2d383-b99a-4d48-bbd9-40d61aac261b None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:34.464589 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 14/28] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:50:34.470228 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11362dd-7384-44d0-a71b-be90f07825a5 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:34.470507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11362dd-7384-44d0-a71b-be90f07825a5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:34.470784 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11362dd-7384-44d0-a71b-be90f07825a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:34.496596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24de9d2-6515-420a-8e51-2df4bebb9560 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:34.510182 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c11362dd-7384-44d0-a71b-be90f07825a5 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:34.511384 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c11362dd-7384-44d0-a71b-be90f07825a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:34.559194 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c84a6-adc3-4e0a-bd63-293dfdc3a5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:34.559884 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c84a6-adc3-4e0a-bd63-293dfdc3a5dc None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:34.560167 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c84a6-adc3-4e0a-bd63-293dfdc3a5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:34.560553 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c84a6-adc3-4e0a-bd63-293dfdc3a5dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:34.587039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c84a6-adc3-4e0a-bd63-293dfdc3a5dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:50:34.592321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-7c3c84a6-adc3-4e0a-bd63-293dfdc3a5dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:50:34.592321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:50:34.592321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:50:34.592321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:50:34.592321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:50:34.592321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:50:34.592321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:50:34.592321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:50:34.592321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:50:34.592321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:50:34.592321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:50:34.592321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:50:34.592321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:50:34.592321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:50:34.592321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:50:34.592321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:50:34.593207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:50:34.593207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:50:34.593207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:50:34.593207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:50:34.593207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:50:34.593207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:50:34.593207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:50:34.593207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:50:34.593207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:50:34.593207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:50:34.593207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:50:34.593207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:50:34.593207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:50:34.593207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:50:34.593207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:50:34.593207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:50:34.593951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:50:34.593951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:50:34.593951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:50:34.593951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:50:34.593951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:50:34.593951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:50:34.593951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:50:34.593951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:50:34.593951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:50:34.593951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:50:34.593951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:50:34.593951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:50:34.602066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 15/29] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:50:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:50:34.602891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c84a6-adc3-4e0a-bd63-293dfdc3a5dc None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:34.615895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:34.637964 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 15/30] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:50:34.640530 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11362dd-7384-44d0-a71b-be90f07825a5 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:34.701060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:34.804614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c84a6-adc3-4e0a-bd63-293dfdc3a5dc None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:34.874797 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b94de1-e06a-45cc-be1a-e8a8b2da8a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:34.875343 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b94de1-e06a-45cc-be1a-e8a8b2da8a85 None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:34.875646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b94de1-e06a-45cc-be1a-e8a8b2da8a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:34.875923 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b94de1-e06a-45cc-be1a-e8a8b2da8a85 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:34.876607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b94de1-e06a-45cc-be1a-e8a8b2da8a85 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:34.878063 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b94de1-e06a-45cc-be1a-e8a8b2da8a85 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:34.879217 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11362dd-7384-44d0-a71b-be90f07825a5 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:34.916274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22532585-5bcf-400e-874f-f5203851b319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:34.916900 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22532585-5bcf-400e-874f-f5203851b319 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:34.917204 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22532585-5bcf-400e-874f-f5203851b319 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:34.917456 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22532585-5bcf-400e-874f-f5203851b319 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:34.918133 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22532585-5bcf-400e-874f-f5203851b319 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:50:34.918431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22532585-5bcf-400e-874f-f5203851b319 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:34.919863 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22532585-5bcf-400e-874f-f5203851b319 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:34.933813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 16/31] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:50:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:34.934762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22532585-5bcf-400e-874f-f5203851b319 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:35.033614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 16/32] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Sep 14 00:50:34 2022] POST /identity/v3/roles => generated 225 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:50:35.035090 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b94de1-e06a-45cc-be1a-e8a8b2da8a85 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:35.046040 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:35.051393 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22532585-5bcf-400e-874f-f5203851b319 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:35.132676 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4973a8-343c-40fa-9472-9d24629f8c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:35.133245 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4973a8-343c-40fa-9472-9d24629f8c4c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:35.133526 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4973a8-343c-40fa-9472-9d24629f8c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:35.133802 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4973a8-343c-40fa-9472-9d24629f8c4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:35.156739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4973a8-343c-40fa-9472-9d24629f8c4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:35.158280 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4973a8-343c-40fa-9472-9d24629f8c4c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:35.167991 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:35.187573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-ed4973a8-343c-40fa-9472-9d24629f8c4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:50:35.187573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:50:35.187573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:50:35.187573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:50:35.187573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:50:35.187573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:50:35.187573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:50:35.187573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:50:35.187573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:50:35.187573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:50:35.187573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:50:35.187573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:50:35.187573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:50:35.187573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:50:35.187573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:50:35.187573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:50:35.188519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:50:35.188519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:50:35.188519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:50:35.188519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:50:35.188519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:50:35.188519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:50:35.188519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:50:35.188519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:50:35.188519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:50:35.188519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:50:35.188519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:50:35.188519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:50:35.188519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:50:35.188519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:50:35.188519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:50:35.188519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:50:35.188519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:50:35.189310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:50:35.189310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:50:35.189310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:50:35.193676 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 17/33] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:50:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:50:35.196473 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4973a8-343c-40fa-9472-9d24629f8c4c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:35.248083 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b94de1-e06a-45cc-be1a-e8a8b2da8a85 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:35.303255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc7ca9-1013-4209-be7f-c6de48d0de92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:35.304373 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc7ca9-1013-4209-be7f-c6de48d0de92 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:35.304860 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc7ca9-1013-4209-be7f-c6de48d0de92 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:35.305337 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc7ca9-1013-4209-be7f-c6de48d0de92 None admin] PATH_INFO: `/v3/users/9dea2911925e405b852c2fe0f9bee81d` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:35.307007 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc7ca9-1013-4209-be7f-c6de48d0de92 None admin] RBAC: Authorizing `identity:get_user(user_id=9dea2911925e405b852c2fe0f9bee81d)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:35.308922 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc7ca9-1013-4209-be7f-c6de48d0de92 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:35.311004 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 17/34] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:50:35 2022] GET /identity/v3/users/9dea2911925e405b852c2fe0f9bee81d => generated 247 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:35.312053 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc7ca9-1013-4209-be7f-c6de48d0de92 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:35.367837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4973a8-343c-40fa-9472-9d24629f8c4c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:35.411811 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31688f96-5bfc-430a-9336-f51e3de462c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:35.412436 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31688f96-5bfc-430a-9336-f51e3de462c2 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:35.412723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31688f96-5bfc-430a-9336-f51e3de462c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:35.413000 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31688f96-5bfc-430a-9336-f51e3de462c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:35.413677 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31688f96-5bfc-430a-9336-f51e3de462c2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:35.415076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31688f96-5bfc-430a-9336-f51e3de462c2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:35.443640 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc7ca9-1013-4209-be7f-c6de48d0de92 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:35.482160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:35.485654 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39130b93-6914-4213-a61c-1a20918db4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:35.486894 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39130b93-6914-4213-a61c-1a20918db4df None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:35.487634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39130b93-6914-4213-a61c-1a20918db4df None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:35.488187 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39130b93-6914-4213-a61c-1a20918db4df None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:35.489269 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39130b93-6914-4213-a61c-1a20918db4df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:50:35.489583 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39130b93-6914-4213-a61c-1a20918db4df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:35.491038 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39130b93-6914-4213-a61c-1a20918db4df None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:35.510501 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 18/35] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:35.511364 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39130b93-6914-4213-a61c-1a20918db4df None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:35.541522 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 18/36] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Sep 14 00:50:35 2022] POST /identity/v3/roles => generated 219 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:50:35.545463 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b05d45-7c07-45b8-966f-1e8affab7003 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:35.545747 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b05d45-7c07-45b8-966f-1e8affab7003 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:35.546028 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b05d45-7c07-45b8-966f-1e8affab7003 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:35.547379 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 19/37] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:50:35.548152 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b05d45-7c07-45b8-966f-1e8affab7003 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:35.569007 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:35.644716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39130b93-6914-4213-a61c-1a20918db4df None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:35.679533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b05d45-7c07-45b8-966f-1e8affab7003 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:35.681342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e507399-4658-476c-8dd2-c8af1b58e966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:35.682004 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e507399-4658-476c-8dd2-c8af1b58e966 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:35.682289 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e507399-4658-476c-8dd2-c8af1b58e966 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:35.682572 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e507399-4658-476c-8dd2-c8af1b58e966 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:35.683869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e507399-4658-476c-8dd2-c8af1b58e966 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:35.685402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e507399-4658-476c-8dd2-c8af1b58e966 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:35.687088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 19/38] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:50:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:35.688422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203b9dc-1fe3-419c-a4b3-1e3b0709af67 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:35.688715 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203b9dc-1fe3-419c-a4b3-1e3b0709af67 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:35.688996 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203b9dc-1fe3-419c-a4b3-1e3b0709af67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:35.710027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d203b9dc-1fe3-419c-a4b3-1e3b0709af67 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:35.711645 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d203b9dc-1fe3-419c-a4b3-1e3b0709af67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:35.727310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6c984-119e-43fb-88e8-ab7e94c8fdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:35.727984 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6c984-119e-43fb-88e8-ab7e94c8fdc3 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:35.728272 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6c984-119e-43fb-88e8-ab7e94c8fdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:35.728894 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6c984-119e-43fb-88e8-ab7e94c8fdc3 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:35.729471 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6c984-119e-43fb-88e8-ab7e94c8fdc3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:35.731904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6c984-119e-43fb-88e8-ab7e94c8fdc3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:35.767224 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 20/39] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 00:50:35 2022] POST /identity/v3/projects => generated 301 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:50:35.768898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd91692-5786-4136-8da6-b3b950dc701c None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:35.769407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd91692-5786-4136-8da6-b3b950dc701c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:35.769892 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd91692-5786-4136-8da6-b3b950dc701c None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:35.771370 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 21/40] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:50:35.772312 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd91692-5786-4136-8da6-b3b950dc701c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:35.778209 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:35.803653 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 20/41] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:50:35.803653 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3649ccb-8a43-43c8-b430-649c32f9562f None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:35.803653 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3649ccb-8a43-43c8-b430-649c32f9562f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:35.803653 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3649ccb-8a43-43c8-b430-649c32f9562f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:35.836456 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-b3649ccb-8a43-43c8-b430-649c32f9562f None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:35.837854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b3649ccb-8a43-43c8-b430-649c32f9562f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:35.848700 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:35.897348 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd91692-5786-4136-8da6-b3b950dc701c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:35.937326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 21/42] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:50:35.946233 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92474ff-5afc-4e70-b2ba-b3812160e831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:35.947086 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92474ff-5afc-4e70-b2ba-b3812160e831 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:35.947379 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92474ff-5afc-4e70-b2ba-b3812160e831 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:35.947700 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92474ff-5afc-4e70-b2ba-b3812160e831 None admin] PATH_INFO: `/v3/domains/default/users/9dea2911925e405b852c2fe0f9bee81d/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:35.950112 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92474ff-5afc-4e70-b2ba-b3812160e831 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=9dea2911925e405b852c2fe0f9bee81d, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:35.951700 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92474ff-5afc-4e70-b2ba-b3812160e831 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:35.969086 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 22/43] 10.222.0.42 () {58 vars in 1361 bytes} [Wed Sep 14 00:50:35 2022] PUT /identity/v3/domains/default/users/9dea2911925e405b852c2fe0f9bee81d/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:50:36.086189 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3649ccb-8a43-43c8-b430-649c32f9562f None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:36.198048 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3649ccb-8a43-43c8-b430-649c32f9562f None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:36.213033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92474ff-5afc-4e70-b2ba-b3812160e831 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:36.321747 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92474ff-5afc-4e70-b2ba-b3812160e831 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:36.331277 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:36.346932 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:36.352354 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d67bb26-3b53-49c4-b1ff-56a4909afa91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:36.353016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67bb26-3b53-49c4-b1ff-56a4909afa91 None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:36.353277 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67bb26-3b53-49c4-b1ff-56a4909afa91 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:36.353555 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67bb26-3b53-49c4-b1ff-56a4909afa91 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:36.354259 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d67bb26-3b53-49c4-b1ff-56a4909afa91 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:36.355696 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d67bb26-3b53-49c4-b1ff-56a4909afa91 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:36.363523 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a149c-4714-4a27-92c3-5498d1c380f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:36.364230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a149c-4714-4a27-92c3-5498d1c380f5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:36.364518 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a149c-4714-4a27-92c3-5498d1c380f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:36.364798 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a149c-4714-4a27-92c3-5498d1c380f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:36.365918 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a149c-4714-4a27-92c3-5498d1c380f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:36.367377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a149c-4714-4a27-92c3-5498d1c380f5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:36.369415 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 23/44] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:50:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:50:36.371011 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 22/45] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Sep 14 00:50:36 2022] POST /identity/v3/roles => generated 223 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:50:36.375643 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a149c-4714-4a27-92c3-5498d1c380f5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:36.494017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a149c-4714-4a27-92c3-5498d1c380f5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:36.532732 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ae833-37ea-4ce4-8cf4-0ef836c692a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:36.533433 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ae833-37ea-4ce4-8cf4-0ef836c692a3 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:36.533719 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ae833-37ea-4ce4-8cf4-0ef836c692a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:36.534697 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ae833-37ea-4ce4-8cf4-0ef836c692a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:36.535470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ae833-37ea-4ce4-8cf4-0ef836c692a3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:36.536993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ae833-37ea-4ce4-8cf4-0ef836c692a3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:36.566749 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 24/46] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 00:50:36 2022] POST /identity/v3/projects => generated 312 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 00:50:38.182944 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b7fd1-525e-48b9-bb88-5fb1665d0ed5 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:38.183501 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b7fd1-525e-48b9-bb88-5fb1665d0ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:38.183501 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b7fd1-525e-48b9-bb88-5fb1665d0ed5 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:38.184891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 23/47] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:50:38.190421 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937c747-1222-4c90-82eb-4719b367f00c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:38.190907 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937c747-1222-4c90-82eb-4719b367f00c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:38.191010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937c747-1222-4c90-82eb-4719b367f00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:38.209393 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b937c747-1222-4c90-82eb-4719b367f00c None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:38.210886 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b937c747-1222-4c90-82eb-4719b367f00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:38.245316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:38.277598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 25/48] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:50:38.544466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b7fd1-525e-48b9-bb88-5fb1665d0ed5 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:38.649860 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b7fd1-525e-48b9-bb88-5fb1665d0ed5 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:38.684236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdd521-3095-44b7-b649-071d75962024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:38.684882 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdd521-3095-44b7-b649-071d75962024 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:38.685170 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdd521-3095-44b7-b649-071d75962024 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:38.685452 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdd521-3095-44b7-b649-071d75962024 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:38.695595 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fdd521-3095-44b7-b649-071d75962024 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:50:38.700439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-85fdd521-3095-44b7-b649-071d75962024 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:50:38.700439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:50:38.700439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:50:38.700439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:50:38.700439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:50:38.700439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:50:38.700439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:50:38.700439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:50:38.700439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:50:38.700439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:50:38.700439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:50:38.700439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:50:38.700439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:50:38.700439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:50:38.700439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:50:38.700439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:50:38.701415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:50:38.701415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:50:38.701415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:50:38.701415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:50:38.701415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:50:38.701415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:50:38.701415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:50:38.701415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:50:38.701415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:50:38.701415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:50:38.701415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:50:38.701415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:50:38.701415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:50:38.701415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:50:38.701415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:50:38.701415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:50:38.702333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:50:38.702333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:50:38.702333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:50:38.702333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:50:38.702333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:50:38.702333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:50:38.702333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:50:38.702333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:50:38.702333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:50:38.702333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:50:38.702333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:50:38.702333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:50:38.702333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 24/49] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:50:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:50:38.707652 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937c747-1222-4c90-82eb-4719b367f00c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:38.714627 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1cb312-5cb6-424b-a99d-f6bb44ea33a5 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:38.714915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1cb312-5cb6-424b-a99d-f6bb44ea33a5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:38.715192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1cb312-5cb6-424b-a99d-f6bb44ea33a5 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:38.716567 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 25/50] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:50:38.721949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d147e9-c43c-47b8-b3f7-eff3e46ce083 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:38.722232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d147e9-c43c-47b8-b3f7-eff3e46ce083 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:38.722509 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d147e9-c43c-47b8-b3f7-eff3e46ce083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:38.740393 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-98d147e9-c43c-47b8-b3f7-eff3e46ce083 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:38.741514 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-98d147e9-c43c-47b8-b3f7-eff3e46ce083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:38.775489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:38.808630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937c747-1222-4c90-82eb-4719b367f00c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:38.810125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 26/51] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:50:38.842540 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed113a52-c324-440e-b7a5-92d2bff1fca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:38.843162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed113a52-c324-440e-b7a5-92d2bff1fca5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:38.843444 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed113a52-c324-440e-b7a5-92d2bff1fca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:38.843750 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed113a52-c324-440e-b7a5-92d2bff1fca5 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:38.844416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed113a52-c324-440e-b7a5-92d2bff1fca5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:50:38.844720 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed113a52-c324-440e-b7a5-92d2bff1fca5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:38.846109 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed113a52-c324-440e-b7a5-92d2bff1fca5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:38.854633 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 26/52] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:50:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:50:38.860366 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d147e9-c43c-47b8-b3f7-eff3e46ce083 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:38.959241 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d147e9-c43c-47b8-b3f7-eff3e46ce083 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:38.993316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20648418-7d30-4ae8-be47-8a642a2b70fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:38.993885 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20648418-7d30-4ae8-be47-8a642a2b70fa None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:38.994173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20648418-7d30-4ae8-be47-8a642a2b70fa None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:38.994455 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20648418-7d30-4ae8-be47-8a642a2b70fa None admin] PATH_INFO: `/v3/users/9dea2911925e405b852c2fe0f9bee81d` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:38.995618 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20648418-7d30-4ae8-be47-8a642a2b70fa None admin] RBAC: Authorizing `identity:get_user(user_id=9dea2911925e405b852c2fe0f9bee81d)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:38.997088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20648418-7d30-4ae8-be47-8a642a2b70fa None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:38.998667 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 27/53] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:50:38 2022] GET /identity/v3/users/9dea2911925e405b852c2fe0f9bee81d => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:39.004849 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed113a52-c324-440e-b7a5-92d2bff1fca5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:39.076994 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20648418-7d30-4ae8-be47-8a642a2b70fa None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:39.120681 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:39.148179 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:39.182878 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20648418-7d30-4ae8-be47-8a642a2b70fa None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:39.213401 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed113a52-c324-440e-b7a5-92d2bff1fca5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:39.215567 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0a017-3944-4d7d-b5b1-84798a8887df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:39.216225 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0a017-3944-4d7d-b5b1-84798a8887df None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:39.216507 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0a017-3944-4d7d-b5b1-84798a8887df None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:39.216782 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0a017-3944-4d7d-b5b1-84798a8887df None admin] PATH_INFO: `/v3/domains/default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:39.217882 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0a017-3944-4d7d-b5b1-84798a8887df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:39.219344 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0a017-3944-4d7d-b5b1-84798a8887df None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:39.220935 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 28/54] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:50:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:50:39.227063 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0a017-3944-4d7d-b5b1-84798a8887df None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:39.246982 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e38aa03-7323-45e7-9ed7-2f04c5bb9eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:39.247579 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38aa03-7323-45e7-9ed7-2f04c5bb9eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:39.247870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38aa03-7323-45e7-9ed7-2f04c5bb9eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:39.248156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38aa03-7323-45e7-9ed7-2f04c5bb9eb3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:39.249265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e38aa03-7323-45e7-9ed7-2f04c5bb9eb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:39.250724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e38aa03-7323-45e7-9ed7-2f04c5bb9eb3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:39.252398 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 27/55] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:50:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:39.258588 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e38aa03-7323-45e7-9ed7-2f04c5bb9eb3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:39.328344 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0a017-3944-4d7d-b5b1-84798a8887df None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:39.336822 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:39.361833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e38aa03-7323-45e7-9ed7-2f04c5bb9eb3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:39.363202 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6c388-522c-4830-860c-3be475bfef2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:39.363663 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:39.363854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6c388-522c-4830-860c-3be475bfef2c None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:39.364139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6c388-522c-4830-860c-3be475bfef2c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:39.364421 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6c388-522c-4830-860c-3be475bfef2c None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:39.365069 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6c388-522c-4830-860c-3be475bfef2c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:39.366702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6c388-522c-4830-860c-3be475bfef2c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:39.396229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 29/56] 10.222.0.42 () {60 vars in 1140 bytes} [Wed Sep 14 00:50:39 2022] POST /identity/v3/projects => generated 298 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 00:50:39.398246 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09b77f-b9a6-4c4e-9947-86d8edbdf4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:39.398948 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09b77f-b9a6-4c4e-9947-86d8edbdf4ad None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:39.399234 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09b77f-b9a6-4c4e-9947-86d8edbdf4ad None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:39.399508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09b77f-b9a6-4c4e-9947-86d8edbdf4ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:39.400277 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef09b77f-b9a6-4c4e-9947-86d8edbdf4ad None admin] RBAC: Adding query filter params (role.id=ada268ea63354a0297070369dfe3c6cc, scope.domain.id=default, user.id=9dea2911925e405b852c2fe0f9bee81d) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:50:39.400586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef09b77f-b9a6-4c4e-9947-86d8edbdf4ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:39.402071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef09b77f-b9a6-4c4e-9947-86d8edbdf4ad None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:39.411796 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 28/57] 10.222.0.42 () {56 vars in 1322 bytes} [Wed Sep 14 00:50:39 2022] GET /identity/v3/role_assignments?user.id=9dea2911925e405b852c2fe0f9bee81d&scope.domain.id=default&role.id=ada268ea63354a0297070369dfe3c6cc => generated 530 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:50:41.576337 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0757f23-277a-4551-a336-a658c0ddb7e1 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:41.576866 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0757f23-277a-4551-a336-a658c0ddb7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:41.576988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0757f23-277a-4551-a336-a658c0ddb7e1 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:41.578342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 30/58] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:50:41.583776 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6caddb4-995a-4b51-be06-8da7c3eb9389 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:41.584191 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6caddb4-995a-4b51-be06-8da7c3eb9389 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:41.584339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6caddb4-995a-4b51-be06-8da7c3eb9389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:41.602669 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c6caddb4-995a-4b51-be06-8da7c3eb9389 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:41.603899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c6caddb4-995a-4b51-be06-8da7c3eb9389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:41.668698 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 29/59] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:50:41.939287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0757f23-277a-4551-a336-a658c0ddb7e1 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:42.040202 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0757f23-277a-4551-a336-a658c0ddb7e1 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:42.072279 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ec760-e011-434e-b32a-13bb35b6442e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:42.072908 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ec760-e011-434e-b32a-13bb35b6442e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:42.073193 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ec760-e011-434e-b32a-13bb35b6442e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:42.073466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ec760-e011-434e-b32a-13bb35b6442e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:42.074562 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ec760-e011-434e-b32a-13bb35b6442e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:42.076045 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ec760-e011-434e-b32a-13bb35b6442e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:42.077796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 31/60] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:50:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:42.084764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6caddb4-995a-4b51-be06-8da7c3eb9389 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:42.175704 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:42.181945 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6caddb4-995a-4b51-be06-8da7c3eb9389 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:42.216949 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf37d1-36a7-4fed-9b35-eef028aff02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:42.217494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf37d1-36a7-4fed-9b35-eef028aff02f None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:42.217776 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf37d1-36a7-4fed-9b35-eef028aff02f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:42.218050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf37d1-36a7-4fed-9b35-eef028aff02f None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:42.218703 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf37d1-36a7-4fed-9b35-eef028aff02f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:42.220167 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf37d1-36a7-4fed-9b35-eef028aff02f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:42.245484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 30/61] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 00:50:42 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:50:45.509896 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d44e5b-7a3c-4dfe-9fde-b947ee165513 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:45.510398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d44e5b-7a3c-4dfe-9fde-b947ee165513 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:45.510505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d44e5b-7a3c-4dfe-9fde-b947ee165513 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:45.511932 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 32/62] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:50:45.517483 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9050c7a-3c82-4a1d-bfe9-061d0e88fdff None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:45.519058 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9050c7a-3c82-4a1d-bfe9-061d0e88fdff None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:45.519675 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9050c7a-3c82-4a1d-bfe9-061d0e88fdff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:45.547773 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a9050c7a-3c82-4a1d-bfe9-061d0e88fdff None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:45.549209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a9050c7a-3c82-4a1d-bfe9-061d0e88fdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:45.646017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 31/63] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:50:45.699438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0ad82-c320-4ad9-866b-f0e7adcd76ae None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:45.699970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0ad82-c320-4ad9-866b-f0e7adcd76ae None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:45.700389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0ad82-c320-4ad9-866b-f0e7adcd76ae None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:45.701905 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 33/64] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:50:45.707662 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4ae07-aeaf-4b5b-8840-cda665ffc9de None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:45.708115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4ae07-aeaf-4b5b-8840-cda665ffc9de None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:45.708531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4ae07-aeaf-4b5b-8840-cda665ffc9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:45.738923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-1ec4ae07-aeaf-4b5b-8840-cda665ffc9de None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:45.740373 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1ec4ae07-aeaf-4b5b-8840-cda665ffc9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:45.837920 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 32/65] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:50:45.934158 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0ad82-c320-4ad9-866b-f0e7adcd76ae None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:45.957268 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4682372-6fee-49df-b23e-9ab5b0d778cd None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:45.957878 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4682372-6fee-49df-b23e-9ab5b0d778cd None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:45.958377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4682372-6fee-49df-b23e-9ab5b0d778cd None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:45.959929 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 33/66] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:50:45.969314 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4880a42-b155-4e83-bad4-be17cc19a354 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:45.969314 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4880a42-b155-4e83-bad4-be17cc19a354 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:45.969314 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4880a42-b155-4e83-bad4-be17cc19a354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:45.974923 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:45.986322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c4880a42-b155-4e83-bad4-be17cc19a354 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:45.988497 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c4880a42-b155-4e83-bad4-be17cc19a354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:45.996651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:46.099722 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0ad82-c320-4ad9-866b-f0e7adcd76ae None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:46.118719 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 34/67] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:50:46.147401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd2245-a46e-4582-8fca-12ef6f6c0c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:46.148095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd2245-a46e-4582-8fca-12ef6f6c0c9f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:46.148514 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd2245-a46e-4582-8fca-12ef6f6c0c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:46.148787 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd2245-a46e-4582-8fca-12ef6f6c0c9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:46.161041 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd2245-a46e-4582-8fca-12ef6f6c0c9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:50:46.166486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-57dd2245-a46e-4582-8fca-12ef6f6c0c9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:50:46.166486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:50:46.166486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:50:46.166486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:50:46.166486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:50:46.166486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:50:46.166486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:50:46.166486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:50:46.166486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:50:46.166486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:50:46.166486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:50:46.166486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:50:46.166486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:50:46.166486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:50:46.166486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:50:46.166486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:50:46.167401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:50:46.167401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:50:46.167401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:50:46.167401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:50:46.167401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:50:46.167401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:50:46.167401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:50:46.167401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:50:46.167401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:50:46.167401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:50:46.167401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:50:46.167401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:50:46.167401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:50:46.167401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:50:46.167401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:50:46.167401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:50:46.168228 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:50:46.168228 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:50:46.168228 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:50:46.168228 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:50:46.168228 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:50:46.168228 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:50:46.168228 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:50:46.168228 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:50:46.168228 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:50:46.168228 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:50:46.168228 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:50:46.168228 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:50:46.171718 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 34/68] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:50:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:50:46.173635 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4880a42-b155-4e83-bad4-be17cc19a354 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:46.191738 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd2245-a46e-4582-8fca-12ef6f6c0c9f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:46.308954 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4880a42-b155-4e83-bad4-be17cc19a354 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:46.310778 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:46.328078 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd2245-a46e-4582-8fca-12ef6f6c0c9f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:46.348251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a089e9c-ff6c-4ddf-9512-d70d563f9a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:46.348876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a089e9c-ff6c-4ddf-9512-d70d563f9a26 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:46.349162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a089e9c-ff6c-4ddf-9512-d70d563f9a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:46.349441 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a089e9c-ff6c-4ddf-9512-d70d563f9a26 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:46.350544 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a089e9c-ff6c-4ddf-9512-d70d563f9a26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:50:46.350856 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a089e9c-ff6c-4ddf-9512-d70d563f9a26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:46.352434 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a089e9c-ff6c-4ddf-9512-d70d563f9a26 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:46.360938 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 35/69] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:50:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:46.365739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc289c-7e64-4a7b-aafb-341c9f9d247f None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:46.365739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc289c-7e64-4a7b-aafb-341c9f9d247f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:46.365739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc289c-7e64-4a7b-aafb-341c9f9d247f None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:46.365739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 36/70] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:50:46.365739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d508182-9244-4064-80d7-820af84d6e63 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:46.365739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d508182-9244-4064-80d7-820af84d6e63 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:46.366094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d508182-9244-4064-80d7-820af84d6e63 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:46.366880 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 37/71] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:50:46.370343 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3504ec-ef1e-4c8a-99d9-c7c479d5bfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:46.371002 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3504ec-ef1e-4c8a-99d9-c7c479d5bfec None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:46.371287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3504ec-ef1e-4c8a-99d9-c7c479d5bfec None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:46.371391 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d508182-9244-4064-80d7-820af84d6e63 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:46.371581 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3504ec-ef1e-4c8a-99d9-c7c479d5bfec None admin] PATH_INFO: `/v3/roles/member` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:46.375817 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:46.383256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3504ec-ef1e-4c8a-99d9-c7c479d5bfec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:50:46.388140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-3c3504ec-ef1e-4c8a-99d9-c7c479d5bfec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:50:46.388140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:50:46.388140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:50:46.388140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:50:46.388140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:50:46.388140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:50:46.388140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:50:46.388140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:50:46.388140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:50:46.388140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:50:46.388140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:50:46.388140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:50:46.388140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:50:46.388140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:50:46.388140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:50:46.388140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:50:46.388983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:50:46.388983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:50:46.388983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:50:46.388983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:50:46.388983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:50:46.388983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:50:46.388983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:50:46.388983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:50:46.388983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:50:46.388983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:50:46.388983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:50:46.388983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:50:46.388983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:50:46.388983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:50:46.388983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:50:46.388983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:50:46.389739 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:50:46.389739 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:50:46.389739 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:50:46.389739 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:50:46.389739 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:50:46.389739 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:50:46.389739 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:50:46.389739 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:50:46.389739 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:50:46.389739 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:50:46.389739 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:50:46.389739 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:50:46.389739 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 35/72] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:50:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:50:46.390629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993bb4a-9282-4725-abd2-105f0237d509 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:46.391645 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993bb4a-9282-4725-abd2-105f0237d509 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:46.391645 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993bb4a-9282-4725-abd2-105f0237d509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:46.410154 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-7993bb4a-9282-4725-abd2-105f0237d509 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:46.411380 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7993bb4a-9282-4725-abd2-105f0237d509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:46.477835 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 36/73] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:50:46.479351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df4d3a-17c2-4ede-89ea-28bb06db0aa2 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:46.479871 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df4d3a-17c2-4ede-89ea-28bb06db0aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:46.480682 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df4d3a-17c2-4ede-89ea-28bb06db0aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:46.497303 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d508182-9244-4064-80d7-820af84d6e63 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:46.498487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-b1df4d3a-17c2-4ede-89ea-28bb06db0aa2 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:46.499710 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b1df4d3a-17c2-4ede-89ea-28bb06db0aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:46.532632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5f348-18e1-4df3-b337-6203ecb7727b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:46.533387 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5f348-18e1-4df3-b337-6203ecb7727b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:46.533688 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5f348-18e1-4df3-b337-6203ecb7727b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:46.533965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5f348-18e1-4df3-b337-6203ecb7727b None admin] PATH_INFO: `/v3/users/9dea2911925e405b852c2fe0f9bee81d` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:46.535261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a5f348-18e1-4df3-b337-6203ecb7727b None admin] RBAC: Authorizing `identity:get_user(user_id=9dea2911925e405b852c2fe0f9bee81d)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:46.536773 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a5f348-18e1-4df3-b337-6203ecb7727b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:46.538570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 38/74] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:50:46 2022] GET /identity/v3/users/9dea2911925e405b852c2fe0f9bee81d => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:46.539130 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5f348-18e1-4df3-b337-6203ecb7727b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:46.579118 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 37/75] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:50:46.581353 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df4d3a-17c2-4ede-89ea-28bb06db0aa2 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:46.590854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:46.625955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:46.666691 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5f348-18e1-4df3-b337-6203ecb7727b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:46.705151 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df4d3a-17c2-4ede-89ea-28bb06db0aa2 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:46.709398 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded40df-93bc-4fce-884e-411dd9fa3f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:46.710011 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded40df-93bc-4fce-884e-411dd9fa3f47 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:46.710295 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded40df-93bc-4fce-884e-411dd9fa3f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:46.710573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded40df-93bc-4fce-884e-411dd9fa3f47 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:46.711235 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ded40df-93bc-4fce-884e-411dd9fa3f47 None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:50:46.711542 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ded40df-93bc-4fce-884e-411dd9fa3f47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:46.712976 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ded40df-93bc-4fce-884e-411dd9fa3f47 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:46.724918 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 39/76] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:50:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:46.725930 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded40df-93bc-4fce-884e-411dd9fa3f47 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:46.751976 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752229b-31e4-4f1a-8715-cd108614e2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:46.753465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752229b-31e4-4f1a-8715-cd108614e2dd None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:46.753465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752229b-31e4-4f1a-8715-cd108614e2dd None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:46.753713 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752229b-31e4-4f1a-8715-cd108614e2dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:46.765798 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752229b-31e4-4f1a-8715-cd108614e2dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:50:46.771131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-8752229b-31e4-4f1a-8715-cd108614e2dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:50:46.771131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:50:46.771131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:50:46.771131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:50:46.771131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:50:46.771131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:50:46.771131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:50:46.771131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:50:46.771131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:50:46.771131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:50:46.771131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:50:46.771131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:50:46.771131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:50:46.771131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:50:46.771131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:50:46.771131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:50:46.772021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:50:46.772021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:50:46.772021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:50:46.772021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:50:46.772021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:50:46.772021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:50:46.772021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:50:46.772021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:50:46.772021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:50:46.772021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:50:46.772021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:50:46.772021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:50:46.772021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:50:46.772021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:50:46.772021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:50:46.772021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:50:46.772793 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:50:46.772793 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:50:46.772793 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:50:46.772793 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:50:46.772793 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:50:46.772793 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:50:46.772793 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:50:46.772793 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:50:46.772793 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:50:46.772793 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:50:46.772793 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:50:46.772793 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:50:46.772793 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 38/77] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 00:50:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:50:46.776396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752229b-31e4-4f1a-8715-cd108614e2dd None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:46.843155 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded40df-93bc-4fce-884e-411dd9fa3f47 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:46.843890 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:46.877289 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057f38d-e7f9-49bb-ad78-f1c9bc6e47e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:46.878887 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057f38d-e7f9-49bb-ad78-f1c9bc6e47e1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:46.879168 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057f38d-e7f9-49bb-ad78-f1c9bc6e47e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:46.879438 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057f38d-e7f9-49bb-ad78-f1c9bc6e47e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:46.881619 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752229b-31e4-4f1a-8715-cd108614e2dd None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:46.882061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:46.915650 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d6ae6-e1bb-489d-bed1-6b772ef07b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:46.916278 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d6ae6-e1bb-489d-bed1-6b772ef07b54 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:46.916558 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d6ae6-e1bb-489d-bed1-6b772ef07b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:46.916830 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d6ae6-e1bb-489d-bed1-6b772ef07b54 None admin] PATH_INFO: `/v3/users/938f59e04b4b4ad7bb6b69e44e819036` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:46.929682 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636d6ae6-e1bb-489d-bed1-6b772ef07b54 None admin] RBAC: Authorizing `identity:get_user(user_id=938f59e04b4b4ad7bb6b69e44e819036)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:46.931156 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636d6ae6-e1bb-489d-bed1-6b772ef07b54 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:46.932795 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 39/78] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:50:46 2022] GET /identity/v3/users/938f59e04b4b4ad7bb6b69e44e819036 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:46.933559 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d6ae6-e1bb-489d-bed1-6b772ef07b54 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:46.948974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1057f38d-e7f9-49bb-ad78-f1c9bc6e47e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:46.950465 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1057f38d-e7f9-49bb-ad78-f1c9bc6e47e1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:47.031494 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d6ae6-e1bb-489d-bed1-6b772ef07b54 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:47.048866 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 40/79] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:50:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:50:47.049893 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057f38d-e7f9-49bb-ad78-f1c9bc6e47e1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:47.054931 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:47.066553 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90631a5-a8e5-48de-963a-fb2e6091fa1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:47.067193 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90631a5-a8e5-48de-963a-fb2e6091fa1d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:47.067482 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90631a5-a8e5-48de-963a-fb2e6091fa1d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:47.067858 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90631a5-a8e5-48de-963a-fb2e6091fa1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:47.078007 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90631a5-a8e5-48de-963a-fb2e6091fa1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:50:47.083758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-d90631a5-a8e5-48de-963a-fb2e6091fa1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:50:47.083758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:50:47.083758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:50:47.083758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:50:47.083758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:50:47.083758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:50:47.083758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:50:47.083758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:50:47.083758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:50:47.083758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:50:47.083758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:50:47.083758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:50:47.083758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:50:47.083758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:50:47.083758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:50:47.083758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:50:47.084680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:50:47.084680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:50:47.084680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:50:47.084680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:50:47.084680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:50:47.084680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:50:47.084680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:50:47.084680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:50:47.084680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:50:47.084680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:50:47.084680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:50:47.084680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:50:47.084680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:50:47.084680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:50:47.084680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:50:47.084680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:50:47.085447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:50:47.085447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:50:47.085447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:50:47.085447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:50:47.085447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:50:47.085447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:50:47.085447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:50:47.085447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:50:47.085447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:50:47.085447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:50:47.085447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:50:47.085447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:50:47.085447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 40/80] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:50:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:50:47.086599 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90631a5-a8e5-48de-963a-fb2e6091fa1d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:47.087042 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:47.153707 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057f38d-e7f9-49bb-ad78-f1c9bc6e47e1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:47.185316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90631a5-a8e5-48de-963a-fb2e6091fa1d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:47.187872 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3396264-4c6f-4935-b9a3-322ebddb9743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:47.188482 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3396264-4c6f-4935-b9a3-322ebddb9743 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:47.188764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3396264-4c6f-4935-b9a3-322ebddb9743 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:47.189039 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3396264-4c6f-4935-b9a3-322ebddb9743 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:47.189726 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3396264-4c6f-4935-b9a3-322ebddb9743 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:50:47.190136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3396264-4c6f-4935-b9a3-322ebddb9743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:47.191963 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3396264-4c6f-4935-b9a3-322ebddb9743 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:47.199754 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 41/81] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:50:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:47.200542 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3396264-4c6f-4935-b9a3-322ebddb9743 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:47.219254 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a654e-d046-45d3-a5f9-ef5b5f431191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:47.219931 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a654e-d046-45d3-a5f9-ef5b5f431191 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:47.220219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a654e-d046-45d3-a5f9-ef5b5f431191 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:47.220496 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a654e-d046-45d3-a5f9-ef5b5f431191 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:47.221779 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855a654e-d046-45d3-a5f9-ef5b5f431191 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:47.223179 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855a654e-d046-45d3-a5f9-ef5b5f431191 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:47.224787 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 41/82] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:50:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:47.225554 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a654e-d046-45d3-a5f9-ef5b5f431191 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:47.278433 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:47.303712 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:47.310050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3396264-4c6f-4935-b9a3-322ebddb9743 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:47.326001 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a654e-d046-45d3-a5f9-ef5b5f431191 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:47.345071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6cbb7-672a-41aa-b366-666014b043d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:47.345691 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6cbb7-672a-41aa-b366-666014b043d8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:47.345987 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6cbb7-672a-41aa-b366-666014b043d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:47.346821 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6cbb7-672a-41aa-b366-666014b043d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:47.361110 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197a311-687c-4529-b281-228fb796b68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:47.362279 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197a311-687c-4529-b281-228fb796b68b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:47.362576 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197a311-687c-4529-b281-228fb796b68b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:47.362860 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197a311-687c-4529-b281-228fb796b68b None admin] PATH_INFO: `/v3/projects/ac5d363867c44c349f5be02f57cd5958` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:47.364034 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7197a311-687c-4529-b281-228fb796b68b None admin] RBAC: Authorizing `identity:get_project(project_id=ac5d363867c44c349f5be02f57cd5958)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:47.365515 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7197a311-687c-4529-b281-228fb796b68b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:47.367162 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 42/83] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:50:47 2022] GET /identity/v3/projects/ac5d363867c44c349f5be02f57cd5958 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:47.368044 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197a311-687c-4529-b281-228fb796b68b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:47.416041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6cbb7-672a-41aa-b366-666014b043d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:47.417525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6cbb7-672a-41aa-b366-666014b043d8 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:47.465276 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197a311-687c-4529-b281-228fb796b68b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:47.493629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:47.508539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2e505-bb81-4a57-838c-9906551fe895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:47.509158 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2e505-bb81-4a57-838c-9906551fe895 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:47.509450 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2e505-bb81-4a57-838c-9906551fe895 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:47.509732 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2e505-bb81-4a57-838c-9906551fe895 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:47.510404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2e505-bb81-4a57-838c-9906551fe895 None admin] RBAC: Adding query filter params (name=member) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:50:47.510714 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2e505-bb81-4a57-838c-9906551fe895 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:47.512074 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:47.512177 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2e505-bb81-4a57-838c-9906551fe895 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:47.520124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 42/84] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:50:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:50:47.520947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6cbb7-672a-41aa-b366-666014b043d8 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:47.521416 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 43/85] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:50:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:47.522212 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2e505-bb81-4a57-838c-9906551fe895 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:47.620795 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6cbb7-672a-41aa-b366-666014b043d8 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:47.621357 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2e505-bb81-4a57-838c-9906551fe895 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:47.653423 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e761d0d-ba1a-433c-93f0-184913ffb96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:47.654033 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e761d0d-ba1a-433c-93f0-184913ffb96b None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:47.654314 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e761d0d-ba1a-433c-93f0-184913ffb96b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:47.654589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e761d0d-ba1a-433c-93f0-184913ffb96b None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:47.654712 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7eaa8d-41a4-4017-8aef-8e8ed32e8511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:47.655237 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e761d0d-ba1a-433c-93f0-184913ffb96b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:47.655334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7eaa8d-41a4-4017-8aef-8e8ed32e8511 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:47.655524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7eaa8d-41a4-4017-8aef-8e8ed32e8511 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:47.655836 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7eaa8d-41a4-4017-8aef-8e8ed32e8511 None admin] PATH_INFO: `/v3/users/938f59e04b4b4ad7bb6b69e44e819036` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:47.656709 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e761d0d-ba1a-433c-93f0-184913ffb96b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:47.656965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7eaa8d-41a4-4017-8aef-8e8ed32e8511 None admin] RBAC: Authorizing `identity:get_user(user_id=938f59e04b4b4ad7bb6b69e44e819036)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:47.658409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7eaa8d-41a4-4017-8aef-8e8ed32e8511 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:47.660558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 43/86] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:50:47 2022] GET /identity/v3/users/938f59e04b4b4ad7bb6b69e44e819036 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:47.661323 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7eaa8d-41a4-4017-8aef-8e8ed32e8511 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:47.684410 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 44/87] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 00:50:47 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:50:47.685606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e761d0d-ba1a-433c-93f0-184913ffb96b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:47.732333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:47.741636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:47.777166 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7eaa8d-41a4-4017-8aef-8e8ed32e8511 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:47.813882 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e761d0d-ba1a-433c-93f0-184913ffb96b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:47.817339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf1f73-7e55-462e-893a-66fc1774ddcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:47.817768 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf1f73-7e55-462e-893a-66fc1774ddcc None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:47.818055 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf1f73-7e55-462e-893a-66fc1774ddcc None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:47.818334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf1f73-7e55-462e-893a-66fc1774ddcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:47.819085 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf1f73-7e55-462e-893a-66fc1774ddcc None admin] RBAC: Adding query filter params (role.id=ada268ea63354a0297070369dfe3c6cc, scope.project.id=ac5d363867c44c349f5be02f57cd5958, user.id=9dea2911925e405b852c2fe0f9bee81d) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:50:47.819403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf1f73-7e55-462e-893a-66fc1774ddcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:47.820925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf1f73-7e55-462e-893a-66fc1774ddcc None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:47.835587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 44/88] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:50:47 2022] GET /identity/v3/role_assignments?user.id=9dea2911925e405b852c2fe0f9bee81d&scope.project.id=ac5d363867c44c349f5be02f57cd5958&role.id=ada268ea63354a0297070369dfe3c6cc => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:47.835587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf1f73-7e55-462e-893a-66fc1774ddcc None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:47.873590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e647a-0738-4460-9bf5-788c390678f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:47.874263 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e647a-0738-4460-9bf5-788c390678f8 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:47.874549 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e647a-0738-4460-9bf5-788c390678f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:47.874827 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e647a-0738-4460-9bf5-788c390678f8 None admin] PATH_INFO: `/v3/projects/ac5d363867c44c349f5be02f57cd5958` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:47.876118 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e647a-0738-4460-9bf5-788c390678f8 None admin] RBAC: Authorizing `identity:get_project(project_id=ac5d363867c44c349f5be02f57cd5958)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:47.878351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e647a-0738-4460-9bf5-788c390678f8 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:47.881289 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 45/89] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:50:47 2022] GET /identity/v3/projects/ac5d363867c44c349f5be02f57cd5958 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:47.881900 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e647a-0738-4460-9bf5-788c390678f8 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:47.984091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf1f73-7e55-462e-893a-66fc1774ddcc None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:48.010162 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e647a-0738-4460-9bf5-788c390678f8 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:48.019787 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:48.023427 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:48.033366 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf4a64-af21-4c49-a94c-3740dc6cebc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:48.034089 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf4a64-af21-4c49-a94c-3740dc6cebc5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:48.034379 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf4a64-af21-4c49-a94c-3740dc6cebc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:48.034660 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf4a64-af21-4c49-a94c-3740dc6cebc5 None admin] PATH_INFO: `/v3/users/938f59e04b4b4ad7bb6b69e44e819036` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:48.036804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf4a64-af21-4c49-a94c-3740dc6cebc5 None admin] RBAC: Authorizing `identity:get_user(user_id=938f59e04b4b4ad7bb6b69e44e819036)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:48.038309 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf4a64-af21-4c49-a94c-3740dc6cebc5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:48.041149 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 45/90] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:50:47 2022] GET /identity/v3/users/938f59e04b4b4ad7bb6b69e44e819036 => generated 275 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:48.042182 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf4a64-af21-4c49-a94c-3740dc6cebc5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:48.058080 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94c7b2-7126-4543-9d0e-44ae7ad7c6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:48.058856 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94c7b2-7126-4543-9d0e-44ae7ad7c6fc None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:48.059141 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94c7b2-7126-4543-9d0e-44ae7ad7c6fc None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:48.059422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94c7b2-7126-4543-9d0e-44ae7ad7c6fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:48.157678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94c7b2-7126-4543-9d0e-44ae7ad7c6fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:48.160154 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94c7b2-7126-4543-9d0e-44ae7ad7c6fc None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:48.176640 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf4a64-af21-4c49-a94c-3740dc6cebc5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:48.220823 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb8db2-f884-4cd3-b854-023547ab6aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:48.222661 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb8db2-f884-4cd3-b854-023547ab6aa0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:48.223150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb8db2-f884-4cd3-b854-023547ab6aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:48.223679 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb8db2-f884-4cd3-b854-023547ab6aa0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:48.224866 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb8db2-f884-4cd3-b854-023547ab6aa0 None admin] RBAC: Adding query filter params (role.id=5d9045e45840408a92ceca71e9184ba2, scope.project.id=ac5d363867c44c349f5be02f57cd5958, user.id=938f59e04b4b4ad7bb6b69e44e819036) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:50:48.225406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb8db2-f884-4cd3-b854-023547ab6aa0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:48.227870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb8db2-f884-4cd3-b854-023547ab6aa0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:48.235317 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 46/91] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:50:48 2022] GET /identity/v3/role_assignments?user.id=938f59e04b4b4ad7bb6b69e44e819036&scope.project.id=ac5d363867c44c349f5be02f57cd5958&role.id=5d9045e45840408a92ceca71e9184ba2 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:48.236577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb8db2-f884-4cd3-b854-023547ab6aa0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:48.277130 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 46/92] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:50:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:50:48.285171 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94c7b2-7126-4543-9d0e-44ae7ad7c6fc None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:48.295169 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:48.314560 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:48.368550 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb8db2-f884-4cd3-b854-023547ab6aa0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:48.414162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9fc61-b3c7-4026-9eac-e48ac5ad1677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:48.415353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9fc61-b3c7-4026-9eac-e48ac5ad1677 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:48.415845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9fc61-b3c7-4026-9eac-e48ac5ad1677 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:48.416411 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9fc61-b3c7-4026-9eac-e48ac5ad1677 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:48.420192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94c7b2-7126-4543-9d0e-44ae7ad7c6fc None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:48.465771 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49342b44-8f9d-4dc3-b1a3-603e972d16aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:48.466487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49342b44-8f9d-4dc3-b1a3-603e972d16aa None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:48.466776 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49342b44-8f9d-4dc3-b1a3-603e972d16aa None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:48.467054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49342b44-8f9d-4dc3-b1a3-603e972d16aa None admin] PATH_INFO: `/v3/projects/ac5d363867c44c349f5be02f57cd5958` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:48.468454 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49342b44-8f9d-4dc3-b1a3-603e972d16aa None admin] RBAC: Authorizing `identity:get_project(project_id=ac5d363867c44c349f5be02f57cd5958)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:48.469948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49342b44-8f9d-4dc3-b1a3-603e972d16aa None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:48.472248 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 47/93] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:50:48 2022] GET /identity/v3/projects/ac5d363867c44c349f5be02f57cd5958 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:48.482075 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49342b44-8f9d-4dc3-b1a3-603e972d16aa None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:48.510676 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9fc61-b3c7-4026-9eac-e48ac5ad1677 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:48.512595 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9fc61-b3c7-4026-9eac-e48ac5ad1677 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:48.585506 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:48.588849 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:48.622500 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49342b44-8f9d-4dc3-b1a3-603e972d16aa None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:48.638150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 47/94] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:50:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:50:48.645085 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9fc61-b3c7-4026-9eac-e48ac5ad1677 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:48.672621 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347a1be-1eff-458a-9f3c-893904e02a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:48.673540 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347a1be-1eff-458a-9f3c-893904e02a1a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:48.673995 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347a1be-1eff-458a-9f3c-893904e02a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:48.674963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347a1be-1eff-458a-9f3c-893904e02a1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:48.676315 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5347a1be-1eff-458a-9f3c-893904e02a1a None admin] RBAC: Adding query filter params (role.id=89a3bc954b0a45bfa40a16db0ad2e51c, scope.project.id=ac5d363867c44c349f5be02f57cd5958, user.id=938f59e04b4b4ad7bb6b69e44e819036) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:50:48.676831 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5347a1be-1eff-458a-9f3c-893904e02a1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:48.678469 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5347a1be-1eff-458a-9f3c-893904e02a1a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:48.684816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 48/95] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:50:48 2022] GET /identity/v3/role_assignments?user.id=938f59e04b4b4ad7bb6b69e44e819036&scope.project.id=ac5d363867c44c349f5be02f57cd5958&role.id=89a3bc954b0a45bfa40a16db0ad2e51c => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:48.759106 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9fc61-b3c7-4026-9eac-e48ac5ad1677 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:48.797715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e40347-a052-4c47-946e-862f626742d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:48.798365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e40347-a052-4c47-946e-862f626742d0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:48.798652 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e40347-a052-4c47-946e-862f626742d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:48.798944 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e40347-a052-4c47-946e-862f626742d0 None admin] PATH_INFO: `/v3/projects/2aed3ce25de843aa9cdac9e9cd162d0b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:48.800451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e40347-a052-4c47-946e-862f626742d0 None admin] RBAC: Authorizing `identity:get_project(project_id=2aed3ce25de843aa9cdac9e9cd162d0b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:48.802008 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e40347-a052-4c47-946e-862f626742d0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:48.803865 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 48/96] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:50:48 2022] GET /identity/v3/projects/2aed3ce25de843aa9cdac9e9cd162d0b => generated 312 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:48.809911 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347a1be-1eff-458a-9f3c-893904e02a1a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:48.943320 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347a1be-1eff-458a-9f3c-893904e02a1a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:48.987385 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c47b6c-4acb-4b47-9140-0fd15c81e051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:48.988033 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c47b6c-4acb-4b47-9140-0fd15c81e051 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:48.988320 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c47b6c-4acb-4b47-9140-0fd15c81e051 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:48.988604 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c47b6c-4acb-4b47-9140-0fd15c81e051 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:48.989343 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c47b6c-4acb-4b47-9140-0fd15c81e051 None admin] RBAC: Adding query filter params (role.id=5d9045e45840408a92ceca71e9184ba2, scope.project.id=2aed3ce25de843aa9cdac9e9cd162d0b, user.id=938f59e04b4b4ad7bb6b69e44e819036) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:50:48.989657 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c47b6c-4acb-4b47-9140-0fd15c81e051 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:48.991117 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c47b6c-4acb-4b47-9140-0fd15c81e051 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:48.997328 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 49/97] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:50:48 2022] GET /identity/v3/role_assignments?user.id=938f59e04b4b4ad7bb6b69e44e819036&scope.project.id=2aed3ce25de843aa9cdac9e9cd162d0b&role.id=5d9045e45840408a92ceca71e9184ba2 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:50.957911 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4e1de-794a-492d-b7b7-9a7da540c74b None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:50.958453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4e1de-794a-492d-b7b7-9a7da540c74b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:50.958453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4e1de-794a-492d-b7b7-9a7da540c74b None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:50.960229 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 49/98] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:50:50.965988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e55da-ed8f-40da-82eb-80edfb56088d None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:50.966457 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e55da-ed8f-40da-82eb-80edfb56088d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:50.966647 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e55da-ed8f-40da-82eb-80edfb56088d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:50.973448 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:50.989177 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-bf7e55da-ed8f-40da-82eb-80edfb56088d None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:50.990381 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bf7e55da-ed8f-40da-82eb-80edfb56088d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:51.067481 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 50/99] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:50:51.092023 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce9940-f3aa-4ab2-989b-013e51779798 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:51.092319 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce9940-f3aa-4ab2-989b-013e51779798 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:51.092598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce9940-f3aa-4ab2-989b-013e51779798 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:51.093943 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 50/100] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:50:51.100180 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49d003-5c88-465d-a675-f60a2c92da71 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:51.100484 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49d003-5c88-465d-a675-f60a2c92da71 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:51.100772 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49d003-5c88-465d-a675-f60a2c92da71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:51.128810 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-3a49d003-5c88-465d-a675-f60a2c92da71 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:51.130031 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-3a49d003-5c88-465d-a675-f60a2c92da71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:51.213933 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 51/101] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:50:51.541423 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce9940-f3aa-4ab2-989b-013e51779798 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:51.579862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:51.595856 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f8f1f-7e53-4d64-afa2-2bc1a741fbd5 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:51.596199 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f8f1f-7e53-4d64-afa2-2bc1a741fbd5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:51.596478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1f8f1f-7e53-4d64-afa2-2bc1a741fbd5 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:51.597842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 52/102] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:50:51.603105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d4bb3-2282-43b5-b21c-965d6e9e33b2 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:51.603398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d4bb3-2282-43b5-b21c-965d6e9e33b2 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:51.603718 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d4bb3-2282-43b5-b21c-965d6e9e33b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:51.623295 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-bb3d4bb3-2282-43b5-b21c-965d6e9e33b2 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:51.624554 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bb3d4bb3-2282-43b5-b21c-965d6e9e33b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:51.669964 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce9940-f3aa-4ab2-989b-013e51779798 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:51.689432 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:51.702370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 53/103] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:50:51.709671 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77e844-31f7-4853-a702-438651f020d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:51.710321 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77e844-31f7-4853-a702-438651f020d7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:51.710606 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77e844-31f7-4853-a702-438651f020d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:51.710883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77e844-31f7-4853-a702-438651f020d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:51.712019 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77e844-31f7-4853-a702-438651f020d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:51.713472 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77e844-31f7-4853-a702-438651f020d7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:51.715033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 51/104] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:50:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:51.720531 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d4bb3-2282-43b5-b21c-965d6e9e33b2 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:51.785969 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77e844-31f7-4853-a702-438651f020d7 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:51.882886 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d4bb3-2282-43b5-b21c-965d6e9e33b2 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:51.921789 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:51.928510 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77e844-31f7-4853-a702-438651f020d7 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:51.970603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab375f64-3625-4b49-84c8-c1c683139e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:51.972023 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab375f64-3625-4b49-84c8-c1c683139e8c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:51.972412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab375f64-3625-4b49-84c8-c1c683139e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:51.972989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab375f64-3625-4b49-84c8-c1c683139e8c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:51.985187 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab375f64-3625-4b49-84c8-c1c683139e8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:50:51.993032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-ab375f64-3625-4b49-84c8-c1c683139e8c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 00:50:51.993032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:50:51.993032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:50:51.993032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:50:51.993032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:50:51.993032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:50:51.993032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:50:51.993032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:50:51.993032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:50:51.993032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:50:51.993032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:50:51.993032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:50:51.993032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:50:51.993032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:50:51.993032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:50:51.993032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:50:51.993915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:50:51.993915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:50:51.993915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:50:51.993915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:50:51.993915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:50:51.993915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:50:51.993915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:50:51.993915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:50:51.993915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:50:51.993915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:50:51.993915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:50:51.993915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:50:51.993915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:50:51.993915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:50:51.993915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:50:51.993915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:50:51.994674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:50:51.994674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:50:51.994674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:50:51.994674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:50:51.994674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:50:51.994674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:50:51.994674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:50:51.994674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:50:51.994674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:50:51.994674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:50:51.994674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 00:50:51.994674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:50:51.994674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 52/105] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:50:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:50:51.999824 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab375f64-3625-4b49-84c8-c1c683139e8c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:52.005825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f65f3-5b2c-4686-b1e1-a05a08c7bae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:52.006997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f65f3-5b2c-4686-b1e1-a05a08c7bae7 None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:52.007474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f65f3-5b2c-4686-b1e1-a05a08c7bae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:52.007968 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f65f3-5b2c-4686-b1e1-a05a08c7bae7 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:52.008808 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f65f3-5b2c-4686-b1e1-a05a08c7bae7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:52.010453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f65f3-5b2c-4686-b1e1-a05a08c7bae7 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:52.074079 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 54/106] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 00:50:51 2022] POST /identity/v3/projects => generated 302 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:50:52.075631 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5efba-224c-4950-a981-f667a8a55a43 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:52.076115 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5efba-224c-4950-a981-f667a8a55a43 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:52.076551 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5efba-224c-4950-a981-f667a8a55a43 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:52.078117 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 55/107] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:50:52.079478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f38ab-31f0-42e6-b538-eb7421c45046 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:52.079999 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f38ab-31f0-42e6-b538-eb7421c45046 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:52.080544 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f38ab-31f0-42e6-b538-eb7421c45046 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:52.083007 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 56/108] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:50:52.084534 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddce6d-9619-4c74-9f11-daac14811ec9 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:52.084997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddce6d-9619-4c74-9f11-daac14811ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:52.085440 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddce6d-9619-4c74-9f11-daac14811ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:52.124140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-08ddce6d-9619-4c74-9f11-daac14811ec9 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:52.126163 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-08ddce6d-9619-4c74-9f11-daac14811ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:52.150686 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab375f64-3625-4b49-84c8-c1c683139e8c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:52.188549 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:52.204428 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eafad2-e89b-4c80-9749-a72d11bac17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:52.205482 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eafad2-e89b-4c80-9749-a72d11bac17f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:52.206527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eafad2-e89b-4c80-9749-a72d11bac17f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:52.207001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eafad2-e89b-4c80-9749-a72d11bac17f None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:52.207900 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eafad2-e89b-4c80-9749-a72d11bac17f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:50:52.208384 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eafad2-e89b-4c80-9749-a72d11bac17f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:52.210776 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eafad2-e89b-4c80-9749-a72d11bac17f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:52.222086 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 57/109] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:50:52.223156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 53/110] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:50:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:50:52.223847 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec4e87-384a-44f7-bbc3-fe55f18d90d4 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:52.224238 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec4e87-384a-44f7-bbc3-fe55f18d90d4 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:52.224592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec4e87-384a-44f7-bbc3-fe55f18d90d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:52.224871 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eafad2-e89b-4c80-9749-a72d11bac17f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:52.244919 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d0ec4e87-384a-44f7-bbc3-fe55f18d90d4 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:52.247303 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d0ec4e87-384a-44f7-bbc3-fe55f18d90d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:52.248921 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:52.335671 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 58/111] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:50:52.336578 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec4e87-384a-44f7-bbc3-fe55f18d90d4 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:52.365363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eafad2-e89b-4c80-9749-a72d11bac17f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:52.411425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199a653-1684-47ae-9c7f-3d34c77d7480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:52.412123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199a653-1684-47ae-9c7f-3d34c77d7480 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:52.413291 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199a653-1684-47ae-9c7f-3d34c77d7480 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:52.413291 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199a653-1684-47ae-9c7f-3d34c77d7480 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:52.426888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199a653-1684-47ae-9c7f-3d34c77d7480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:50:52.432353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-2199a653-1684-47ae-9c7f-3d34c77d7480 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:50:52.432353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:50:52.432353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:50:52.432353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:50:52.432353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:50:52.432353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:50:52.432353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:50:52.432353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:50:52.432353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:50:52.432353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:50:52.432353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:50:52.432353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:50:52.432353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:50:52.432353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:50:52.432353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:50:52.432353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:50:52.433417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:50:52.433417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:50:52.433417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:50:52.433417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:50:52.433417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:50:52.433417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:50:52.433417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:50:52.433417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:50:52.433417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:50:52.433417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:50:52.433417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:50:52.433417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:50:52.433417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:50:52.433417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:50:52.433417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:50:52.433417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:50:52.434363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:50:52.434363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:50:52.434363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:50:52.434363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:50:52.434363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:50:52.434363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:50:52.434363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:50:52.434363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:50:52.434363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:50:52.434363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:50:52.434363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:50:52.434363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:50:52.435737 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 54/112] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:50:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:50:52.436897 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4ad87-af9e-49c8-836a-efde95a5f3eb None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:52.437255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4ad87-af9e-49c8-836a-efde95a5f3eb None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:52.437597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4ad87-af9e-49c8-836a-efde95a5f3eb None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:52.439993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 55/113] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:50:52.446980 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e436a0-3633-41e2-ad0e-4466fcbe4be8 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:52.447390 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e436a0-3633-41e2-ad0e-4466fcbe4be8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:52.447802 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e436a0-3633-41e2-ad0e-4466fcbe4be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:52.460807 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec4e87-384a-44f7-bbc3-fe55f18d90d4 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:52.463367 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:52.489673 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-24e436a0-3633-41e2-ad0e-4466fcbe4be8 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:52.492458 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-24e436a0-3633-41e2-ad0e-4466fcbe4be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:52.515544 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b56837-aec6-47e7-9bc3-f936dd423f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:52.516313 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b56837-aec6-47e7-9bc3-f936dd423f78 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:52.516701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b56837-aec6-47e7-9bc3-f936dd423f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:52.516982 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b56837-aec6-47e7-9bc3-f936dd423f78 None admin] PATH_INFO: `/v3/users/7228f12915de43ab88e029a4d42ebd93` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:52.542395 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b56837-aec6-47e7-9bc3-f936dd423f78 None admin] RBAC: Authorizing `identity:get_user(user_id=7228f12915de43ab88e029a4d42ebd93)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:52.543924 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b56837-aec6-47e7-9bc3-f936dd423f78 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:52.547318 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 59/114] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:50:52 2022] GET /identity/v3/users/7228f12915de43ab88e029a4d42ebd93 => generated 289 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:52.548156 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b56837-aec6-47e7-9bc3-f936dd423f78 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:52.606902 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 56/115] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:50:52.607538 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e436a0-3633-41e2-ad0e-4466fcbe4be8 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:52.622189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:52.684968 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b56837-aec6-47e7-9bc3-f936dd423f78 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:52.730277 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc791dcc-b2c1-46b0-8ea8-84765e20d366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:52.730900 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc791dcc-b2c1-46b0-8ea8-84765e20d366 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:52.731187 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc791dcc-b2c1-46b0-8ea8-84765e20d366 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:52.731468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc791dcc-b2c1-46b0-8ea8-84765e20d366 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:52.732366 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc791dcc-b2c1-46b0-8ea8-84765e20d366 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:50:52.732677 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc791dcc-b2c1-46b0-8ea8-84765e20d366 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:52.734727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc791dcc-b2c1-46b0-8ea8-84765e20d366 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:52.744041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e436a0-3633-41e2-ad0e-4466fcbe4be8 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:52.745413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 60/116] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:50:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:52.746086 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc791dcc-b2c1-46b0-8ea8-84765e20d366 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:52.779749 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:52.784776 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed664a-df75-4a7b-9314-7e40813ff043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:52.785409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed664a-df75-4a7b-9314-7e40813ff043 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:52.785700 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed664a-df75-4a7b-9314-7e40813ff043 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:52.785983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed664a-df75-4a7b-9314-7e40813ff043 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:52.859531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed664a-df75-4a7b-9314-7e40813ff043 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:52.860987 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed664a-df75-4a7b-9314-7e40813ff043 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:52.871566 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:52.888648 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc791dcc-b2c1-46b0-8ea8-84765e20d366 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:52.926418 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb85af-217e-4fe6-9dde-d8d4969a8a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:52.927057 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb85af-217e-4fe6-9dde-d8d4969a8a63 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:52.927342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb85af-217e-4fe6-9dde-d8d4969a8a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:52.927659 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb85af-217e-4fe6-9dde-d8d4969a8a63 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:52.937970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb85af-217e-4fe6-9dde-d8d4969a8a63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:50:52.944116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-3dcb85af-217e-4fe6-9dde-d8d4969a8a63 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:50:52.944116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:50:52.944116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:50:52.944116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:50:52.944116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:50:52.944116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:50:52.944116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:50:52.944116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:50:52.944116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:50:52.944116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:50:52.944116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:50:52.944116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:50:52.944116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:50:52.944116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:50:52.944116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:50:52.944116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:50:52.945050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:50:52.945050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:50:52.945050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:50:52.945050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:50:52.945050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:50:52.945050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:50:52.945050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:50:52.945050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:50:52.945050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:50:52.945050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:50:52.945050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:50:52.945050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:50:52.945050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:50:52.945050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:50:52.945050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:50:52.945050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:50:52.945819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:50:52.945819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:50:52.945819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:50:52.945819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:50:52.945819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:50:52.945819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:50:52.945819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:50:52.945819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:50:52.945819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:50:52.945819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:50:52.945819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:50:52.945819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:50:52.945819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 61/117] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:50:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:50:52.946450 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb85af-217e-4fe6-9dde-d8d4969a8a63 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:52.974171 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 57/118] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:50:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:50:52.979329 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed664a-df75-4a7b-9314-7e40813ff043 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:53.043347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:53.060811 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb85af-217e-4fe6-9dde-d8d4969a8a63 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:53.089388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed664a-df75-4a7b-9314-7e40813ff043 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:53.097070 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b0f6f-5140-4852-9b29-6e2139517211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:53.097722 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b0f6f-5140-4852-9b29-6e2139517211 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:53.098012 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b0f6f-5140-4852-9b29-6e2139517211 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:53.098294 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b0f6f-5140-4852-9b29-6e2139517211 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:53.099965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:53.110320 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b0f6f-5140-4852-9b29-6e2139517211 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:50:53.115759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-b51b0f6f-5140-4852-9b29-6e2139517211 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:50:53.115759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:50:53.115759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:50:53.115759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:50:53.115759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:50:53.115759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:50:53.115759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:50:53.115759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:50:53.115759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:50:53.115759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:50:53.115759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:50:53.115759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:50:53.115759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:50:53.115759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:50:53.115759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:50:53.115759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:50:53.116614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:50:53.116614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:50:53.116614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:50:53.116614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:50:53.116614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:50:53.116614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:50:53.116614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:50:53.116614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:50:53.116614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:50:53.116614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:50:53.116614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:50:53.116614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:50:53.116614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:50:53.116614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:50:53.116614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:50:53.116614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:50:53.117388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:50:53.117388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:50:53.117388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:50:53.117388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:50:53.117388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:50:53.117388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:50:53.117388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:50:53.117388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:50:53.117388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:50:53.117388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:50:53.117388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:50:53.117388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:50:53.117388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 62/119] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 00:50:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:50:53.118009 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b0f6f-5140-4852-9b29-6e2139517211 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:53.135871 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd75e30-c12a-443a-9c18-9cdae5fe2aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:53.136886 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd75e30-c12a-443a-9c18-9cdae5fe2aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:53.137316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd75e30-c12a-443a-9c18-9cdae5fe2aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:53.137597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd75e30-c12a-443a-9c18-9cdae5fe2aa9 None admin] PATH_INFO: `/v3/users/9dea2911925e405b852c2fe0f9bee81d` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:53.138749 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd75e30-c12a-443a-9c18-9cdae5fe2aa9 None admin] RBAC: Authorizing `identity:get_user(user_id=9dea2911925e405b852c2fe0f9bee81d)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:53.140498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd75e30-c12a-443a-9c18-9cdae5fe2aa9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:53.147787 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 58/120] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:50:52 2022] GET /identity/v3/users/9dea2911925e405b852c2fe0f9bee81d => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:53.147787 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd75e30-c12a-443a-9c18-9cdae5fe2aa9 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:53.234646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b0f6f-5140-4852-9b29-6e2139517211 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:53.257447 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd75e30-c12a-443a-9c18-9cdae5fe2aa9 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:53.274320 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213a563-a3f4-4bd2-9242-81ff628982f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:53.275453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213a563-a3f4-4bd2-9242-81ff628982f9 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:53.275775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213a563-a3f4-4bd2-9242-81ff628982f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:53.276057 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213a563-a3f4-4bd2-9242-81ff628982f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:53.288233 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8213a563-a3f4-4bd2-9242-81ff628982f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:50:53.292267 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79596555-315c-4629-af82-e8e8a6d6dff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:53.292877 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79596555-315c-4629-af82-e8e8a6d6dff1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:53.293162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79596555-315c-4629-af82-e8e8a6d6dff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:53.293440 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79596555-315c-4629-af82-e8e8a6d6dff1 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:53.293661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-8213a563-a3f4-4bd2-9242-81ff628982f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:50:53.293661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:50:53.293661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:50:53.293661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:50:53.293661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:50:53.293661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:50:53.293661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:50:53.293661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:50:53.293661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:50:53.293661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:50:53.293661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:50:53.293661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:50:53.293661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:50:53.293661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:50:53.293661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:50:53.293661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:50:53.294474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:50:53.294474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:50:53.294474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:50:53.294474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:50:53.294474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:50:53.294474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:50:53.294474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:50:53.294474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:50:53.294474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:50:53.294474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:50:53.294474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:50:53.294474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:50:53.294474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:50:53.294474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:50:53.294474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:50:53.294474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:50:53.295256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:50:53.295256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:50:53.295256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:50:53.295256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:50:53.295256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:50:53.295256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:50:53.295256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:50:53.295256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:50:53.295256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:50:53.295256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:50:53.295256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:50:53.295256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:50:53.295858 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79596555-315c-4629-af82-e8e8a6d6dff1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:50:53.295858 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79596555-315c-4629-af82-e8e8a6d6dff1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:53.295991 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 63/121] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:50:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:50:53.295991 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213a563-a3f4-4bd2-9242-81ff628982f9 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:53.300626 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:53.300817 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79596555-315c-4629-af82-e8e8a6d6dff1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:53.314406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 59/122] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:50:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:53.314406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79596555-315c-4629-af82-e8e8a6d6dff1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:53.361380 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:53.414411 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213a563-a3f4-4bd2-9242-81ff628982f9 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:53.429062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79596555-315c-4629-af82-e8e8a6d6dff1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:53.448530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee44a95-8878-4a80-a61a-122b2e7ac353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:53.449173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee44a95-8878-4a80-a61a-122b2e7ac353 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:53.449491 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee44a95-8878-4a80-a61a-122b2e7ac353 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:53.449736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee44a95-8878-4a80-a61a-122b2e7ac353 None admin] PATH_INFO: `/v3/projects/ac5d363867c44c349f5be02f57cd5958` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:53.450862 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee44a95-8878-4a80-a61a-122b2e7ac353 None admin] RBAC: Authorizing `identity:get_project(project_id=ac5d363867c44c349f5be02f57cd5958)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:53.452352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee44a95-8878-4a80-a61a-122b2e7ac353 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:53.458909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 64/123] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:50:53 2022] GET /identity/v3/projects/ac5d363867c44c349f5be02f57cd5958 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:53.458909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee44a95-8878-4a80-a61a-122b2e7ac353 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:53.470057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c3bca-da4a-458a-9355-e6ad04f8a9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:53.470683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c3bca-da4a-458a-9355-e6ad04f8a9ab None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:53.470966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c3bca-da4a-458a-9355-e6ad04f8a9ab None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:53.471801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c3bca-da4a-458a-9355-e6ad04f8a9ab None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:53.472479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c3bca-da4a-458a-9355-e6ad04f8a9ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:50:53.472901 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c3bca-da4a-458a-9355-e6ad04f8a9ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:53.474405 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c3bca-da4a-458a-9355-e6ad04f8a9ab None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:53.483469 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 60/124] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:50:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:53.484386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c3bca-da4a-458a-9355-e6ad04f8a9ab None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:53.542997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:53.575121 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee44a95-8878-4a80-a61a-122b2e7ac353 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:53.592120 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:53.595524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c3bca-da4a-458a-9355-e6ad04f8a9ab None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:53.612910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef16a38-82d9-4ace-a43b-be9b435f0350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:53.613643 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef16a38-82d9-4ace-a43b-be9b435f0350 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:53.614206 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef16a38-82d9-4ace-a43b-be9b435f0350 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:53.614506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef16a38-82d9-4ace-a43b-be9b435f0350 None admin] PATH_INFO: `/v3/projects/ac5d363867c44c349f5be02f57cd5958` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:53.615685 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef16a38-82d9-4ace-a43b-be9b435f0350 None admin] RBAC: Authorizing `identity:get_project(project_id=ac5d363867c44c349f5be02f57cd5958)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:53.617600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef16a38-82d9-4ace-a43b-be9b435f0350 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:53.624302 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 65/125] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:50:53 2022] GET /identity/v3/projects/ac5d363867c44c349f5be02f57cd5958 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:53.624302 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef16a38-82d9-4ace-a43b-be9b435f0350 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:53.631666 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c8d18-a05e-4f00-ae9b-1f7575149e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:53.632291 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c8d18-a05e-4f00-ae9b-1f7575149e10 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:53.632580 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c8d18-a05e-4f00-ae9b-1f7575149e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:53.632862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c8d18-a05e-4f00-ae9b-1f7575149e10 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:53.633538 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c8d18-a05e-4f00-ae9b-1f7575149e10 None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:50:53.633850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c8d18-a05e-4f00-ae9b-1f7575149e10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:53.635271 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c8d18-a05e-4f00-ae9b-1f7575149e10 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:53.649140 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 61/126] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:50:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:53.649140 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c8d18-a05e-4f00-ae9b-1f7575149e10 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:53.734640 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef16a38-82d9-4ace-a43b-be9b435f0350 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:53.751414 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c8d18-a05e-4f00-ae9b-1f7575149e10 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:53.773214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b93a51-5073-4ef5-a64a-22108e5772af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:53.773868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b93a51-5073-4ef5-a64a-22108e5772af None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:53.774192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b93a51-5073-4ef5-a64a-22108e5772af None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:53.774479 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b93a51-5073-4ef5-a64a-22108e5772af None admin] PATH_INFO: `/v3/users/938f59e04b4b4ad7bb6b69e44e819036` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:53.775659 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b93a51-5073-4ef5-a64a-22108e5772af None admin] RBAC: Authorizing `identity:get_user(user_id=938f59e04b4b4ad7bb6b69e44e819036)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:53.777123 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b93a51-5073-4ef5-a64a-22108e5772af None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:53.778774 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 66/127] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:50:53 2022] GET /identity/v3/users/938f59e04b4b4ad7bb6b69e44e819036 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:53.779545 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b93a51-5073-4ef5-a64a-22108e5772af None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:53.790895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:53.793349 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e6453-080b-42f1-91a1-03411cb362f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:53.794347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e6453-080b-42f1-91a1-03411cb362f9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:53.794777 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e6453-080b-42f1-91a1-03411cb362f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:53.795924 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e6453-080b-42f1-91a1-03411cb362f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:53.796924 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e6453-080b-42f1-91a1-03411cb362f9 None admin] RBAC: Adding query filter params (role.id=f330ff772cca47c4b89a406784e00800, scope.project.id=ac5d363867c44c349f5be02f57cd5958, user.id=7228f12915de43ab88e029a4d42ebd93) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:50:53.797380 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e6453-080b-42f1-91a1-03411cb362f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:53.799121 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e6453-080b-42f1-91a1-03411cb362f9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:53.811595 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 62/128] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:50:53 2022] GET /identity/v3/role_assignments?user.id=7228f12915de43ab88e029a4d42ebd93&scope.project.id=ac5d363867c44c349f5be02f57cd5958&role.id=f330ff772cca47c4b89a406784e00800 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:53.811595 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e6453-080b-42f1-91a1-03411cb362f9 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:53.862652 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:53.896798 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b93a51-5073-4ef5-a64a-22108e5772af None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:53.935087 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cac7b9-d69f-46db-9c2c-c301fedb3251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:53.935798 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cac7b9-d69f-46db-9c2c-c301fedb3251 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:53.936083 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cac7b9-d69f-46db-9c2c-c301fedb3251 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:53.936359 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cac7b9-d69f-46db-9c2c-c301fedb3251 None admin] PATH_INFO: `/v3/users/938f59e04b4b4ad7bb6b69e44e819036` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:53.937538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cac7b9-d69f-46db-9c2c-c301fedb3251 None admin] RBAC: Authorizing `identity:get_user(user_id=938f59e04b4b4ad7bb6b69e44e819036)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:53.939139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cac7b9-d69f-46db-9c2c-c301fedb3251 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:53.941638 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 67/129] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:50:53 2022] GET /identity/v3/users/938f59e04b4b4ad7bb6b69e44e819036 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:53.942625 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cac7b9-d69f-46db-9c2c-c301fedb3251 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:53.979803 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e6453-080b-42f1-91a1-03411cb362f9 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:54.023286 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4acf6-7cef-42b4-99f8-c6a05006526a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:54.024318 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4acf6-7cef-42b4-99f8-c6a05006526a None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:54.024609 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4acf6-7cef-42b4-99f8-c6a05006526a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:54.024941 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4acf6-7cef-42b4-99f8-c6a05006526a None admin] PATH_INFO: `/v3/projects/ac5d363867c44c349f5be02f57cd5958/users/9dea2911925e405b852c2fe0f9bee81d/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:54.028118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4acf6-7cef-42b4-99f8-c6a05006526a None admin] RBAC: Authorizing `identity:create_grant(project_id=ac5d363867c44c349f5be02f57cd5958, user_id=9dea2911925e405b852c2fe0f9bee81d, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:54.029602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4acf6-7cef-42b4-99f8-c6a05006526a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:54.033966 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:54.048684 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 63/130] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 00:50:53 2022] PUT /identity/v3/projects/ac5d363867c44c349f5be02f57cd5958/users/9dea2911925e405b852c2fe0f9bee81d/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:50:54.049939 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4acf6-7cef-42b4-99f8-c6a05006526a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:54.070966 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cac7b9-d69f-46db-9c2c-c301fedb3251 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:54.119702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796648b-7c8d-447b-81a0-d9bc6fdeae67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:54.120444 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796648b-7c8d-447b-81a0-d9bc6fdeae67 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:54.120732 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796648b-7c8d-447b-81a0-d9bc6fdeae67 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:54.121009 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796648b-7c8d-447b-81a0-d9bc6fdeae67 None admin] PATH_INFO: `/v3/users/938f59e04b4b4ad7bb6b69e44e819036` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:54.122199 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a796648b-7c8d-447b-81a0-d9bc6fdeae67 None admin] RBAC: Authorizing `identity:get_user(user_id=938f59e04b4b4ad7bb6b69e44e819036)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:54.123863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a796648b-7c8d-447b-81a0-d9bc6fdeae67 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:54.125879 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 68/131] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:50:53 2022] GET /identity/v3/users/938f59e04b4b4ad7bb6b69e44e819036 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:54.126792 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796648b-7c8d-447b-81a0-d9bc6fdeae67 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:54.165786 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:54.188021 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4acf6-7cef-42b4-99f8-c6a05006526a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:54.238407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ec613-6b07-4572-b2a6-d066757e5f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:54.239361 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ec613-6b07-4572-b2a6-d066757e5f23 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:54.239682 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ec613-6b07-4572-b2a6-d066757e5f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:54.239971 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ec613-6b07-4572-b2a6-d066757e5f23 None admin] PATH_INFO: `/v3/projects/ac5d363867c44c349f5be02f57cd5958` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:54.242123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ec613-6b07-4572-b2a6-d066757e5f23 None admin] RBAC: Authorizing `identity:get_project(project_id=ac5d363867c44c349f5be02f57cd5958)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:54.243650 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ec613-6b07-4572-b2a6-d066757e5f23 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:54.243972 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796648b-7c8d-447b-81a0-d9bc6fdeae67 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:54.245742 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 64/132] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:50:54 2022] GET /identity/v3/projects/ac5d363867c44c349f5be02f57cd5958 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:54.246434 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ec613-6b07-4572-b2a6-d066757e5f23 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:54.289903 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa0c36-7f85-49b2-9763-0fdbec0525eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:54.290575 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa0c36-7f85-49b2-9763-0fdbec0525eb None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:54.290858 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa0c36-7f85-49b2-9763-0fdbec0525eb None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:54.291138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa0c36-7f85-49b2-9763-0fdbec0525eb None admin] PATH_INFO: `/v3/projects/ac5d363867c44c349f5be02f57cd5958` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:54.292418 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa0c36-7f85-49b2-9763-0fdbec0525eb None admin] RBAC: Authorizing `identity:get_project(project_id=ac5d363867c44c349f5be02f57cd5958)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:54.294158 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa0c36-7f85-49b2-9763-0fdbec0525eb None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:54.296149 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 69/133] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:50:54 2022] GET /identity/v3/projects/ac5d363867c44c349f5be02f57cd5958 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:54.297059 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa0c36-7f85-49b2-9763-0fdbec0525eb None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:54.316066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:54.388850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ec613-6b07-4572-b2a6-d066757e5f23 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:54.418481 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa0c36-7f85-49b2-9763-0fdbec0525eb None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:54.436898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e1de9-75eb-4d74-a833-d46161ebba54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:54.437847 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e1de9-75eb-4d74-a833-d46161ebba54 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:54.438138 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e1de9-75eb-4d74-a833-d46161ebba54 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:54.438428 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e1de9-75eb-4d74-a833-d46161ebba54 None admin] PATH_INFO: `/v3/projects/2aed3ce25de843aa9cdac9e9cd162d0b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:54.439615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493e1de9-75eb-4d74-a833-d46161ebba54 None admin] RBAC: Authorizing `identity:get_project(project_id=2aed3ce25de843aa9cdac9e9cd162d0b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:54.442205 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493e1de9-75eb-4d74-a833-d46161ebba54 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:54.444457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 65/134] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:50:54 2022] GET /identity/v3/projects/2aed3ce25de843aa9cdac9e9cd162d0b => generated 312 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:54.445220 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e1de9-75eb-4d74-a833-d46161ebba54 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:54.457369 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a1418-0b4e-4ea7-8bda-48b8b95b5bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:54.458157 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a1418-0b4e-4ea7-8bda-48b8b95b5bc2 None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:54.458546 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a1418-0b4e-4ea7-8bda-48b8b95b5bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:54.458853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a1418-0b4e-4ea7-8bda-48b8b95b5bc2 None admin] PATH_INFO: `/v3/projects/ac5d363867c44c349f5be02f57cd5958/users/938f59e04b4b4ad7bb6b69e44e819036/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:54.462049 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a1418-0b4e-4ea7-8bda-48b8b95b5bc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac5d363867c44c349f5be02f57cd5958, user_id=938f59e04b4b4ad7bb6b69e44e819036, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:54.463655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a1418-0b4e-4ea7-8bda-48b8b95b5bc2 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:54.481042 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:54.504050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 70/135] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 00:50:54 2022] PUT /identity/v3/projects/ac5d363867c44c349f5be02f57cd5958/users/938f59e04b4b4ad7bb6b69e44e819036/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:50:54.504957 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a1418-0b4e-4ea7-8bda-48b8b95b5bc2 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:54.586511 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e1de9-75eb-4d74-a833-d46161ebba54 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:54.610521 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:54.663874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a1418-0b4e-4ea7-8bda-48b8b95b5bc2 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:54.677120 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ea470-b63c-4769-b394-014e9cf89b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:54.677738 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ea470-b63c-4769-b394-014e9cf89b7d None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:54.678027 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ea470-b63c-4769-b394-014e9cf89b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:54.678307 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ea470-b63c-4769-b394-014e9cf89b7d None admin] PATH_INFO: `/v3/projects/ac5d363867c44c349f5be02f57cd5958/users/938f59e04b4b4ad7bb6b69e44e819036/roles/89a3bc954b0a45bfa40a16db0ad2e51c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:54.684468 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ea470-b63c-4769-b394-014e9cf89b7d None admin] RBAC: Authorizing `identity:create_grant(project_id=ac5d363867c44c349f5be02f57cd5958, user_id=938f59e04b4b4ad7bb6b69e44e819036, role_id=89a3bc954b0a45bfa40a16db0ad2e51c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:54.686006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ea470-b63c-4769-b394-014e9cf89b7d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:54.708817 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d4e4d-cc1e-4789-915e-c5246ef2aad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:54.709937 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d4e4d-cc1e-4789-915e-c5246ef2aad6 None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:54.710442 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d4e4d-cc1e-4789-915e-c5246ef2aad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:54.710990 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d4e4d-cc1e-4789-915e-c5246ef2aad6 None admin] PATH_INFO: `/v3/projects/2aed3ce25de843aa9cdac9e9cd162d0b/users/938f59e04b4b4ad7bb6b69e44e819036/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:54.713558 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d4e4d-cc1e-4789-915e-c5246ef2aad6 None admin] RBAC: Authorizing `identity:create_grant(project_id=2aed3ce25de843aa9cdac9e9cd162d0b, user_id=938f59e04b4b4ad7bb6b69e44e819036, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:54.715589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64d4e4d-cc1e-4789-915e-c5246ef2aad6 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:54.731710 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 71/136] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 00:50:54 2022] PUT /identity/v3/projects/2aed3ce25de843aa9cdac9e9cd162d0b/users/938f59e04b4b4ad7bb6b69e44e819036/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:50:54.733408 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabee520-d150-4e3e-9ab5-19182b50c17a None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:54.733895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabee520-d150-4e3e-9ab5-19182b50c17a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:54.734116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 66/137] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 00:50:54 2022] PUT /identity/v3/projects/ac5d363867c44c349f5be02f57cd5958/users/938f59e04b4b4ad7bb6b69e44e819036/roles/89a3bc954b0a45bfa40a16db0ad2e51c => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:50:54.734661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabee520-d150-4e3e-9ab5-19182b50c17a None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:54.736444 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 72/138] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:50:54.749262 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70028d30-c5aa-4896-986f-a2ed9123ef54 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:54.749567 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70028d30-c5aa-4896-986f-a2ed9123ef54 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:54.749832 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70028d30-c5aa-4896-986f-a2ed9123ef54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:54.771377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-70028d30-c5aa-4896-986f-a2ed9123ef54 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:54.772652 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-70028d30-c5aa-4896-986f-a2ed9123ef54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:54.869985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 67/139] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:50:55.227499 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabee520-d150-4e3e-9ab5-19182b50c17a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:55.368392 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabee520-d150-4e3e-9ab5-19182b50c17a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:55.417325 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:55.422518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b31148-0837-4f94-b48b-a1517152fdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:55.423177 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b31148-0837-4f94-b48b-a1517152fdf7 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:55.423466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b31148-0837-4f94-b48b-a1517152fdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:55.423773 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b31148-0837-4f94-b48b-a1517152fdf7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:55.425561 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b31148-0837-4f94-b48b-a1517152fdf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:55.427476 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b31148-0837-4f94-b48b-a1517152fdf7 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:55.431713 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 73/140] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:50:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:55.434976 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70028d30-c5aa-4896-986f-a2ed9123ef54 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:55.456213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:55.565838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70028d30-c5aa-4896-986f-a2ed9123ef54 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:55.611054 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e30fe3-9ea5-4d88-9f14-038f78e01518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:55.612269 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e30fe3-9ea5-4d88-9f14-038f78e01518 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:55.612559 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e30fe3-9ea5-4d88-9f14-038f78e01518 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:55.612872 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e30fe3-9ea5-4d88-9f14-038f78e01518 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:55.613521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e30fe3-9ea5-4d88-9f14-038f78e01518 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:55.615058 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e30fe3-9ea5-4d88-9f14-038f78e01518 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:55.649796 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 68/141] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 00:50:55 2022] POST /identity/v3/users => generated 283 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 00:50:56.918842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaefe13-db10-4c41-aef9-924b48a45d8c None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:56.919353 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaefe13-db10-4c41-aef9-924b48a45d8c None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:56.919438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaefe13-db10-4c41-aef9-924b48a45d8c None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:56.920787 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 74/142] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:50:56.926834 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b51f1-8162-428e-b590-ca09c1defad0 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:56.927245 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b51f1-8162-428e-b590-ca09c1defad0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:56.927430 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b51f1-8162-428e-b590-ca09c1defad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:56.956318 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-eb4b51f1-8162-428e-b590-ca09c1defad0 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:56.957508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-eb4b51f1-8162-428e-b590-ca09c1defad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:57.041753 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:57.047450 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 69/143] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:50:57.260014 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53c671-a21c-4d12-92b8-4259065c727e None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:57.260260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53c671-a21c-4d12-92b8-4259065c727e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:57.260497 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b53c671-a21c-4d12-92b8-4259065c727e None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:57.261984 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 75/144] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:50:57.277574 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a13b61-88e4-46e2-b16e-6a564b3fde2d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:57.277948 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a13b61-88e4-46e2-b16e-6a564b3fde2d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:57.278234 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a13b61-88e4-46e2-b16e-6a564b3fde2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:57.305420 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f7a13b61-88e4-46e2-b16e-6a564b3fde2d None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:57.306681 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f7a13b61-88e4-46e2-b16e-6a564b3fde2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:57.386305 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 70/145] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:50:57.550164 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b53c671-a21c-4d12-92b8-4259065c727e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:57.676168 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b53c671-a21c-4d12-92b8-4259065c727e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:57.712929 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74bd06-8534-463d-bf55-a3aad83b025e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:57.713583 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74bd06-8534-463d-bf55-a3aad83b025e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:57.713868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74bd06-8534-463d-bf55-a3aad83b025e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:57.714147 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74bd06-8534-463d-bf55-a3aad83b025e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:57.729252 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74bd06-8534-463d-bf55-a3aad83b025e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:50:57.734698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-3b74bd06-8534-463d-bf55-a3aad83b025e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 00:50:57.734698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:50:57.734698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:50:57.734698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:50:57.734698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:50:57.734698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:50:57.734698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:50:57.734698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:50:57.734698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:50:57.734698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:50:57.734698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:50:57.734698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:50:57.734698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:50:57.734698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:50:57.734698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:50:57.734698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:50:57.735634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:50:57.735634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:50:57.735634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:50:57.735634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:50:57.735634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:50:57.735634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:50:57.735634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:50:57.735634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:50:57.735634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:50:57.735634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:50:57.735634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:50:57.735634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:50:57.735634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:50:57.735634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:50:57.735634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:50:57.735634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:50:57.737346 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:50:57.737346 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:50:57.737346 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:50:57.737346 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:50:57.737346 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:50:57.737346 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:50:57.737346 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:50:57.737346 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:50:57.737346 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:50:57.737346 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:50:57.737346 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 00:50:57.737346 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:50:57.737346 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 76/146] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:50:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:50:57.742523 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a13b61-88e4-46e2-b16e-6a564b3fde2d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:57.851579 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a13b61-88e4-46e2-b16e-6a564b3fde2d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:57.884001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:57.889359 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395bb1c-dd1e-484b-9a36-76cf76f2ef4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:57.889997 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395bb1c-dd1e-484b-9a36-76cf76f2ef4f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:57.890540 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395bb1c-dd1e-484b-9a36-76cf76f2ef4f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:57.890818 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395bb1c-dd1e-484b-9a36-76cf76f2ef4f None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:57.891507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395bb1c-dd1e-484b-9a36-76cf76f2ef4f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:50:57.891862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395bb1c-dd1e-484b-9a36-76cf76f2ef4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:57.894262 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395bb1c-dd1e-484b-9a36-76cf76f2ef4f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:57.903574 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 71/147] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:50:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:57.921361 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74bd06-8534-463d-bf55-a3aad83b025e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:57.990724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395bb1c-dd1e-484b-9a36-76cf76f2ef4f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:58.012987 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:58.096104 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74bd06-8534-463d-bf55-a3aad83b025e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:58.136316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395bb1c-dd1e-484b-9a36-76cf76f2ef4f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:58.151380 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd63b7-11f3-4207-add5-8dae82c193c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:58.152307 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd63b7-11f3-4207-add5-8dae82c193c9 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:58.152683 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd63b7-11f3-4207-add5-8dae82c193c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:58.153057 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd63b7-11f3-4207-add5-8dae82c193c9 None admin] PATH_INFO: `/v3/users/7228f12915de43ab88e029a4d42ebd93` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:58.155501 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd63b7-11f3-4207-add5-8dae82c193c9 None admin] RBAC: Authorizing `identity:get_user(user_id=7228f12915de43ab88e029a4d42ebd93)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:58.157183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd63b7-11f3-4207-add5-8dae82c193c9 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:58.159771 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 77/148] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:50:57 2022] GET /identity/v3/users/7228f12915de43ab88e029a4d42ebd93 => generated 289 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:58.161180 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45b900-909c-4243-9526-b3808c2aba4e None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:58.161656 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45b900-909c-4243-9526-b3808c2aba4e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:58.162134 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45b900-909c-4243-9526-b3808c2aba4e None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:58.167323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 78/149] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:50:58.168810 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45b900-909c-4243-9526-b3808c2aba4e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:58.181011 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1262911-801f-43ec-9d67-764fb5960ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:58.181921 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1262911-801f-43ec-9d67-764fb5960ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:58.182336 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1262911-801f-43ec-9d67-764fb5960ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:58.182760 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1262911-801f-43ec-9d67-764fb5960ec1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:58.202713 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1262911-801f-43ec-9d67-764fb5960ec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:50:58.209507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-b1262911-801f-43ec-9d67-764fb5960ec1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:50:58.209507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:50:58.209507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:50:58.209507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:50:58.209507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:50:58.209507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:50:58.209507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:50:58.209507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:50:58.209507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:50:58.209507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:50:58.209507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:50:58.209507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:50:58.209507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:50:58.209507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:50:58.209507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:50:58.209507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:50:58.210383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:50:58.210383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:50:58.210383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:50:58.210383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:50:58.210383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:50:58.210383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:50:58.210383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:50:58.210383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:50:58.210383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:50:58.210383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:50:58.210383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:50:58.210383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:50:58.210383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:50:58.210383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:50:58.210383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:50:58.210383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:50:58.211146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:50:58.211146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:50:58.211146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:50:58.211146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:50:58.211146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:50:58.211146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:50:58.211146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:50:58.211146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:50:58.211146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:50:58.211146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:50:58.211146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:50:58.211146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:50:58.213307 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 72/150] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:50:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:50:58.214848 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319cd03-ad9b-4e58-b4cf-45a0a299a11a None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:58.215385 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319cd03-ad9b-4e58-b4cf-45a0a299a11a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:58.215939 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319cd03-ad9b-4e58-b4cf-45a0a299a11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:58.236875 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-8319cd03-ad9b-4e58-b4cf-45a0a299a11a None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:58.238999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8319cd03-ad9b-4e58-b4cf-45a0a299a11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:58.332100 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:58.372930 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 73/151] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:50:58.373494 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45b900-909c-4243-9526-b3808c2aba4e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:58.373799 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319cd03-ad9b-4e58-b4cf-45a0a299a11a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:58.428356 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:58.448767 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8bce56-a8a8-4fed-85f4-09fe810d7587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:58.449429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8bce56-a8a8-4fed-85f4-09fe810d7587 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:58.449711 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8bce56-a8a8-4fed-85f4-09fe810d7587 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:58.449988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8bce56-a8a8-4fed-85f4-09fe810d7587 None admin] PATH_INFO: `/v3/projects/ac5d363867c44c349f5be02f57cd5958` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:58.451139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8bce56-a8a8-4fed-85f4-09fe810d7587 None admin] RBAC: Authorizing `identity:get_project(project_id=ac5d363867c44c349f5be02f57cd5958)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:58.452659 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8bce56-a8a8-4fed-85f4-09fe810d7587 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:58.454305 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 79/152] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:50:58 2022] GET /identity/v3/projects/ac5d363867c44c349f5be02f57cd5958 => generated 298 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:58.455987 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a753f18-7df2-4579-8f4e-511fcf734178 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:58.456408 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a753f18-7df2-4579-8f4e-511fcf734178 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:58.456569 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a753f18-7df2-4579-8f4e-511fcf734178 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:58.457907 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 80/153] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:50:58.469583 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc80db9-9182-4082-8ece-8f9ade000ef9 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:58.469880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc80db9-9182-4082-8ece-8f9ade000ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:58.470142 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc80db9-9182-4082-8ece-8f9ade000ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:58.508563 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-6cc80db9-9182-4082-8ece-8f9ade000ef9 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:58.509760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6cc80db9-9182-4082-8ece-8f9ade000ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:58.573977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319cd03-ad9b-4e58-b4cf-45a0a299a11a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:58.627444 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 81/154] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:50:58.628327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc80db9-9182-4082-8ece-8f9ade000ef9 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:58.637738 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670c67f-8029-4bfc-9a83-7a9ba2e9dcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:58.638674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670c67f-8029-4bfc-9a83-7a9ba2e9dcee None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:58.639624 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670c67f-8029-4bfc-9a83-7a9ba2e9dcee None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:58.640356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670c67f-8029-4bfc-9a83-7a9ba2e9dcee None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:58.641620 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670c67f-8029-4bfc-9a83-7a9ba2e9dcee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:50:58.642394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670c67f-8029-4bfc-9a83-7a9ba2e9dcee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:58.644847 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670c67f-8029-4bfc-9a83-7a9ba2e9dcee None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:58.661320 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 74/155] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:50:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:58.668236 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670c67f-8029-4bfc-9a83-7a9ba2e9dcee None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:58.761562 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:58.798882 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc80db9-9182-4082-8ece-8f9ade000ef9 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:58.852818 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:58.874981 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f98343-532a-4cdb-b6e4-fe97dfb705d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:58.876024 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f98343-532a-4cdb-b6e4-fe97dfb705d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:58.876492 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f98343-532a-4cdb-b6e4-fe97dfb705d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:58.876946 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f98343-532a-4cdb-b6e4-fe97dfb705d2 None admin] PATH_INFO: `/v3/projects/ac5d363867c44c349f5be02f57cd5958/users/7228f12915de43ab88e029a4d42ebd93/roles/f330ff772cca47c4b89a406784e00800` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:58.879628 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f98343-532a-4cdb-b6e4-fe97dfb705d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac5d363867c44c349f5be02f57cd5958, user_id=7228f12915de43ab88e029a4d42ebd93, role_id=f330ff772cca47c4b89a406784e00800)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:58.881296 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f98343-532a-4cdb-b6e4-fe97dfb705d2 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:58.887578 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670c67f-8029-4bfc-9a83-7a9ba2e9dcee None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:58.915671 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 82/156] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 00:50:58 2022] PUT /identity/v3/projects/ac5d363867c44c349f5be02f57cd5958/users/7228f12915de43ab88e029a4d42ebd93/roles/f330ff772cca47c4b89a406784e00800 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:50:58.948679 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4758041-dfd5-4d57-bf5d-e400d165baa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:58.949354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4758041-dfd5-4d57-bf5d-e400d165baa8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:58.949641 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4758041-dfd5-4d57-bf5d-e400d165baa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:58.949922 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4758041-dfd5-4d57-bf5d-e400d165baa8 None admin] PATH_INFO: `/v3/users/9dea2911925e405b852c2fe0f9bee81d` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:58.952108 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4758041-dfd5-4d57-bf5d-e400d165baa8 None admin] RBAC: Authorizing `identity:get_user(user_id=9dea2911925e405b852c2fe0f9bee81d)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:58.953692 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4758041-dfd5-4d57-bf5d-e400d165baa8 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:58.955481 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 75/157] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:50:58 2022] GET /identity/v3/users/9dea2911925e405b852c2fe0f9bee81d => generated 247 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:50:58.961305 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f98343-532a-4cdb-b6e4-fe97dfb705d2 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:59.009988 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab917f1-b058-4c36-9310-55b2daaeccf8 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:59.010947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab917f1-b058-4c36-9310-55b2daaeccf8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:59.011427 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab917f1-b058-4c36-9310-55b2daaeccf8 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:59.013164 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 76/158] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:50:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:50:59.041310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd87fb-8bf3-4ef3-9db4-988ed23ebddf None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:59.042003 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd87fb-8bf3-4ef3-9db4-988ed23ebddf None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:59.042535 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd87fb-8bf3-4ef3-9db4-988ed23ebddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:59.074838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2efd87fb-8bf3-4ef3-9db4-988ed23ebddf None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:50:59.076428 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2efd87fb-8bf3-4ef3-9db4-988ed23ebddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:50:59.153457 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f98343-532a-4cdb-b6e4-fe97dfb705d2 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:59.212890 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 77/159] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:50:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:50:59.214121 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd87fb-8bf3-4ef3-9db4-988ed23ebddf None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:59.228294 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd369169-15b8-414e-9ece-1302a84c3515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:59.229199 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd369169-15b8-414e-9ece-1302a84c3515 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:59.229621 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd369169-15b8-414e-9ece-1302a84c3515 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:59.230021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd369169-15b8-414e-9ece-1302a84c3515 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:59.271088 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:59.341537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:59.358910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd369169-15b8-414e-9ece-1302a84c3515 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:59.360460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd369169-15b8-414e-9ece-1302a84c3515 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:59.384266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efd87fb-8bf3-4ef3-9db4-988ed23ebddf None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:59.460364 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68161e-2c42-483c-b1a7-183b911e3dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:59.461262 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68161e-2c42-483c-b1a7-183b911e3dff None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:59.461710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68161e-2c42-483c-b1a7-183b911e3dff None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:59.462197 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68161e-2c42-483c-b1a7-183b911e3dff None admin] PATH_INFO: `/v3/roles/member` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:59.491269 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd68161e-2c42-483c-b1a7-183b911e3dff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:50:59.496199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-bd68161e-2c42-483c-b1a7-183b911e3dff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:50:59.496199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:50:59.496199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:50:59.496199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:50:59.496199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:50:59.496199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:50:59.496199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:50:59.496199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:50:59.496199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:50:59.496199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:50:59.496199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:50:59.496199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:50:59.496199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:50:59.496199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:50:59.496199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:50:59.496199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:50:59.497144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:50:59.497144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:50:59.497144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:50:59.497144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:50:59.497144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:50:59.497144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:50:59.497144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:50:59.497144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:50:59.497144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:50:59.497144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:50:59.497144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:50:59.497144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:50:59.497144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:50:59.497144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:50:59.497144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:50:59.497144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:50:59.497939 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:50:59.497939 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:50:59.497939 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:50:59.497939 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:50:59.497939 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:50:59.497939 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:50:59.497939 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:50:59.497939 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:50:59.497939 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:50:59.497939 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:50:59.497939 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:50:59.497939 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:50:59.508051 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 78/160] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:50:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:50:59.509017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68161e-2c42-483c-b1a7-183b911e3dff None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:59.558051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 83/161] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:50:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:50:59.558670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd369169-15b8-414e-9ece-1302a84c3515 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:59.698917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd369169-15b8-414e-9ece-1302a84c3515 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:59.708582 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:59.756497 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:50:59.764010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68161e-2c42-483c-b1a7-183b911e3dff None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:50:59.784113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c246bd-5c6e-4a61-b2b4-586a4cbbb9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:59.784745 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c246bd-5c6e-4a61-b2b4-586a4cbbb9eb None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:59.785036 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c246bd-5c6e-4a61-b2b4-586a4cbbb9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:59.785318 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c246bd-5c6e-4a61-b2b4-586a4cbbb9eb None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:59.786174 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c246bd-5c6e-4a61-b2b4-586a4cbbb9eb None admin] RBAC: Adding query filter params (name=member) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:50:59.786620 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c246bd-5c6e-4a61-b2b4-586a4cbbb9eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:50:59.788237 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c246bd-5c6e-4a61-b2b4-586a4cbbb9eb None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:50:59.804226 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 84/162] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:50:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:50:59.805074 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c246bd-5c6e-4a61-b2b4-586a4cbbb9eb None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:50:59.852353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91a477-eb52-421f-96fc-bf45771b5641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:50:59.853246 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91a477-eb52-421f-96fc-bf45771b5641 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:50:59.853677 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91a477-eb52-421f-96fc-bf45771b5641 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:50:59.854097 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91a477-eb52-421f-96fc-bf45771b5641 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:50:59.884456 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91a477-eb52-421f-96fc-bf45771b5641 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:50:59.889558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-3f91a477-eb52-421f-96fc-bf45771b5641 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:50:59.889558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:50:59.889558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:50:59.889558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:50:59.889558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:50:59.889558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:50:59.889558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:50:59.889558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:50:59.889558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:50:59.889558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:50:59.889558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:50:59.889558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:50:59.889558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:50:59.889558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:50:59.889558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:50:59.889558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:50:59.890914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:50:59.890914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:50:59.890914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:50:59.890914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:50:59.890914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:50:59.890914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:50:59.890914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:50:59.890914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:50:59.890914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:50:59.890914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:50:59.890914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:50:59.890914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:50:59.890914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:50:59.890914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:50:59.890914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:50:59.890914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:50:59.891923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:50:59.891923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:50:59.891923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:50:59.891923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:50:59.891923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:50:59.891923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:50:59.891923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:50:59.891923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:50:59.891923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:50:59.891923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:50:59.891923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:50:59.891923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:50:59.894147 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 79/163] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 00:50:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:50:59.899875 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91a477-eb52-421f-96fc-bf45771b5641 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:00.010879 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c246bd-5c6e-4a61-b2b4-586a4cbbb9eb None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:00.093286 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e24b6d-e25c-4b4f-bd59-bfcfc9f38683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:00.093924 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e24b6d-e25c-4b4f-bd59-bfcfc9f38683 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:00.094207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e24b6d-e25c-4b4f-bd59-bfcfc9f38683 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:00.094560 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e24b6d-e25c-4b4f-bd59-bfcfc9f38683 None admin] PATH_INFO: `/v3/projects/ac5d363867c44c349f5be02f57cd5958` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:00.097018 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e24b6d-e25c-4b4f-bd59-bfcfc9f38683 None admin] RBAC: Authorizing `identity:get_project(project_id=ac5d363867c44c349f5be02f57cd5958)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:00.098556 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e24b6d-e25c-4b4f-bd59-bfcfc9f38683 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:00.100255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 85/164] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:50:59 2022] GET /identity/v3/projects/ac5d363867c44c349f5be02f57cd5958 => generated 298 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:00.101020 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e24b6d-e25c-4b4f-bd59-bfcfc9f38683 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:00.106682 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91a477-eb52-421f-96fc-bf45771b5641 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:00.147355 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:00.192277 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5113a-20af-4a7c-a2d8-56f911ac3b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:00.192936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5113a-20af-4a7c-a2d8-56f911ac3b5f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:00.193216 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5113a-20af-4a7c-a2d8-56f911ac3b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:00.193490 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5113a-20af-4a7c-a2d8-56f911ac3b5f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:00.219827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5113a-20af-4a7c-a2d8-56f911ac3b5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:00.225340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-aaf5113a-20af-4a7c-a2d8-56f911ac3b5f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:00.225340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:00.225340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:00.225340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:00.225340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:00.225340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:00.225340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:00.225340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:00.225340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:00.225340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:00.225340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:00.225340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:00.225340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:00.225340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:00.225340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:00.225340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:00.227395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:00.227395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:00.227395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:00.227395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:00.227395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:00.227395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:00.227395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:00.227395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:00.227395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:00.227395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:00.227395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:00.227395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:00.227395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:00.227395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:00.227395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:00.227395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:00.229298 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:00.229298 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:00.229298 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:00.229298 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:00.229298 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:00.229298 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:00.229298 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:00.229298 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:00.229298 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:00.229298 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:00.229298 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:00.229298 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:00.229298 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 80/165] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:50:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:00.238034 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5113a-20af-4a7c-a2d8-56f911ac3b5f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:00.283982 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:00.329948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e24b6d-e25c-4b4f-bd59-bfcfc9f38683 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:00.379451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5113a-20af-4a7c-a2d8-56f911ac3b5f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:00.418148 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcac351-eb83-415b-ba60-a9f004adf364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:00.419223 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcac351-eb83-415b-ba60-a9f004adf364 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:00.419756 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcac351-eb83-415b-ba60-a9f004adf364 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:00.420191 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcac351-eb83-415b-ba60-a9f004adf364 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:00.421021 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcac351-eb83-415b-ba60-a9f004adf364 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:00.421492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcac351-eb83-415b-ba60-a9f004adf364 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:00.423056 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcac351-eb83-415b-ba60-a9f004adf364 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:00.433282 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 81/166] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:51:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:00.434504 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcac351-eb83-415b-ba60-a9f004adf364 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:00.488308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c36f1d-c8f1-44ca-8bf2-b663a87d56ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:00.488973 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c36f1d-c8f1-44ca-8bf2-b663a87d56ac None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:00.489449 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c36f1d-c8f1-44ca-8bf2-b663a87d56ac None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:00.489876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c36f1d-c8f1-44ca-8bf2-b663a87d56ac None admin] PATH_INFO: `/v3/users/938f59e04b4b4ad7bb6b69e44e819036` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:00.494979 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c36f1d-c8f1-44ca-8bf2-b663a87d56ac None admin] RBAC: Authorizing `identity:get_user(user_id=938f59e04b4b4ad7bb6b69e44e819036)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:00.496673 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c36f1d-c8f1-44ca-8bf2-b663a87d56ac None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:00.498446 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 86/167] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:51:00 2022] GET /identity/v3/users/938f59e04b4b4ad7bb6b69e44e819036 => generated 275 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:00.499376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c36f1d-c8f1-44ca-8bf2-b663a87d56ac None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:00.539928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:00.546045 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcac351-eb83-415b-ba60-a9f004adf364 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:00.587270 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818310ff-2a4a-47b0-9067-fdc6e80d6015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:00.588283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818310ff-2a4a-47b0-9067-fdc6e80d6015 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:00.588789 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818310ff-2a4a-47b0-9067-fdc6e80d6015 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:00.589264 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818310ff-2a4a-47b0-9067-fdc6e80d6015 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:00.590208 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818310ff-2a4a-47b0-9067-fdc6e80d6015 None admin] RBAC: Adding query filter params (role.id=ada268ea63354a0297070369dfe3c6cc, scope.project.id=ac5d363867c44c349f5be02f57cd5958, user.id=9dea2911925e405b852c2fe0f9bee81d) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:00.590721 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818310ff-2a4a-47b0-9067-fdc6e80d6015 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:00.592423 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818310ff-2a4a-47b0-9067-fdc6e80d6015 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:00.599633 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 82/168] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:51:00 2022] GET /identity/v3/role_assignments?user.id=9dea2911925e405b852c2fe0f9bee81d&scope.project.id=ac5d363867c44c349f5be02f57cd5958&role.id=ada268ea63354a0297070369dfe3c6cc => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:00.607596 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca81b6-9992-43bc-904a-e902e61df3ab None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:00.609365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca81b6-9992-43bc-904a-e902e61df3ab None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:00.609859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca81b6-9992-43bc-904a-e902e61df3ab None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:00.612454 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 83/169] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:00 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:00.613696 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca81b6-9992-43bc-904a-e902e61df3ab None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:00.734423 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:00.760775 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca81b6-9992-43bc-904a-e902e61df3ab None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:00.769816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c36f1d-c8f1-44ca-8bf2-b663a87d56ac None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:00.802514 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4623df-fc19-4891-b601-da89a863bbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:00.803536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4623df-fc19-4891-b601-da89a863bbc3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:00.804148 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4623df-fc19-4891-b601-da89a863bbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:00.804630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4623df-fc19-4891-b601-da89a863bbc3 None admin] PATH_INFO: `/v3/users/938f59e04b4b4ad7bb6b69e44e819036` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:00.806092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4623df-fc19-4891-b601-da89a863bbc3 None admin] RBAC: Authorizing `identity:get_user(user_id=938f59e04b4b4ad7bb6b69e44e819036)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:00.807831 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4623df-fc19-4891-b601-da89a863bbc3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:00.810010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 84/170] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:51:00 2022] GET /identity/v3/users/938f59e04b4b4ad7bb6b69e44e819036 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:00.811131 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4623df-fc19-4891-b601-da89a863bbc3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:00.828791 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:00.851928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ddecae-91b1-473b-bb5a-7ef36bab4c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:00.852951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddecae-91b1-473b-bb5a-7ef36bab4c03 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:00.853417 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddecae-91b1-473b-bb5a-7ef36bab4c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:00.853959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddecae-91b1-473b-bb5a-7ef36bab4c03 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:00.854820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ddecae-91b1-473b-bb5a-7ef36bab4c03 None admin] RBAC: Adding query filter params (name=member) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:00.855304 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ddecae-91b1-473b-bb5a-7ef36bab4c03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:00.856906 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ddecae-91b1-473b-bb5a-7ef36bab4c03 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:00.875221 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 87/171] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:51:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:00.877069 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f145b5-67a9-4820-b988-362f19521bfe None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:00.878304 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f145b5-67a9-4820-b988-362f19521bfe None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:00.878764 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f145b5-67a9-4820-b988-362f19521bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:00.919815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-53f145b5-67a9-4820-b988-362f19521bfe None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:00.921403 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-53f145b5-67a9-4820-b988-362f19521bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:00.933123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4623df-fc19-4891-b601-da89a863bbc3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:00.972532 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56314df7-0632-4a07-ad38-2557739acb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:00.973414 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56314df7-0632-4a07-ad38-2557739acb6a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:00.973933 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56314df7-0632-4a07-ad38-2557739acb6a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:00.974353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56314df7-0632-4a07-ad38-2557739acb6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:01.022526 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 88/172] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:01.023067 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f145b5-67a9-4820-b988-362f19521bfe None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:01.062468 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56314df7-0632-4a07-ad38-2557739acb6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:01.063991 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56314df7-0632-4a07-ad38-2557739acb6a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:01.103101 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:01.172442 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:01.204187 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f145b5-67a9-4820-b988-362f19521bfe None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:01.205663 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 85/173] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:51:01.207530 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92e2ba-9d49-46b2-ba4a-ae939d03d3f3 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:01.208063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92e2ba-9d49-46b2-ba4a-ae939d03d3f3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:01.208636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92e2ba-9d49-46b2-ba4a-ae939d03d3f3 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:01.210448 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 86/174] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:01.211383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92e2ba-9d49-46b2-ba4a-ae939d03d3f3 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:01.247948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0be27d-bea9-4cc4-826e-c9ea86a8232a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:01.248564 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0be27d-bea9-4cc4-826e-c9ea86a8232a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:01.248846 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0be27d-bea9-4cc4-826e-c9ea86a8232a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:01.249118 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0be27d-bea9-4cc4-826e-c9ea86a8232a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:01.321897 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92e2ba-9d49-46b2-ba4a-ae939d03d3f3 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:01.327940 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0be27d-bea9-4cc4-826e-c9ea86a8232a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:01.329425 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0be27d-bea9-4cc4-826e-c9ea86a8232a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:01.339512 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:01.363646 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c43e95-165a-4e78-8a59-03013097c405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:01.364288 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c43e95-165a-4e78-8a59-03013097c405 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:01.364574 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c43e95-165a-4e78-8a59-03013097c405 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:01.364853 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c43e95-165a-4e78-8a59-03013097c405 None admin] PATH_INFO: `/v3/users/938f59e04b4b4ad7bb6b69e44e819036` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:01.365991 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c43e95-165a-4e78-8a59-03013097c405 None admin] RBAC: Authorizing `identity:get_user(user_id=938f59e04b4b4ad7bb6b69e44e819036)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:01.367456 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c43e95-165a-4e78-8a59-03013097c405 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:01.369259 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 87/175] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:51:01 2022] GET /identity/v3/users/938f59e04b4b4ad7bb6b69e44e819036 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:01.370173 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109471a7-17b4-421a-afd8-9759f178d076 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:01.370458 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109471a7-17b4-421a-afd8-9759f178d076 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:01.370733 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109471a7-17b4-421a-afd8-9759f178d076 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:01.371955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 88/176] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:01.372653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109471a7-17b4-421a-afd8-9759f178d076 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:01.419966 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:01.446485 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 89/177] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:51:01.447831 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba98d27-e8db-47e2-b9df-bbe4fd213037 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:01.448122 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba98d27-e8db-47e2-b9df-bbe4fd213037 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:01.448398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba98d27-e8db-47e2-b9df-bbe4fd213037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:01.470136 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-cba98d27-e8db-47e2-b9df-bbe4fd213037 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:01.470136 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-cba98d27-e8db-47e2-b9df-bbe4fd213037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:01.497879 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109471a7-17b4-421a-afd8-9759f178d076 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:01.535284 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630bf8b-b3ad-4dd3-acb1-49990ae5c7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:01.536056 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630bf8b-b3ad-4dd3-acb1-49990ae5c7bd None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:01.536340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630bf8b-b3ad-4dd3-acb1-49990ae5c7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:01.536620 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630bf8b-b3ad-4dd3-acb1-49990ae5c7bd None admin] PATH_INFO: `/v3/projects/ac5d363867c44c349f5be02f57cd5958` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:01.537757 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630bf8b-b3ad-4dd3-acb1-49990ae5c7bd None admin] RBAC: Authorizing `identity:get_project(project_id=ac5d363867c44c349f5be02f57cd5958)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:01.539223 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630bf8b-b3ad-4dd3-acb1-49990ae5c7bd None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:01.547263 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 90/178] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:01.547428 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 89/179] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:01 2022] GET /identity/v3/projects/ac5d363867c44c349f5be02f57cd5958 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:01.547521 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba98d27-e8db-47e2-b9df-bbe4fd213037 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:01.547647 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630bf8b-b3ad-4dd3-acb1-49990ae5c7bd None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:01.603651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:01.668344 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630bf8b-b3ad-4dd3-acb1-49990ae5c7bd None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:01.688489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba98d27-e8db-47e2-b9df-bbe4fd213037 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:01.699035 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:01.712070 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7194d74-58c5-4025-8c68-f8438a1f35f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:01.712920 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7194d74-58c5-4025-8c68-f8438a1f35f9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:01.713356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7194d74-58c5-4025-8c68-f8438a1f35f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:01.713770 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7194d74-58c5-4025-8c68-f8438a1f35f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:01.729789 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2056c4e7-b401-49a9-bf7e-194827dff403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:01.730498 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056c4e7-b401-49a9-bf7e-194827dff403 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:01.730795 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056c4e7-b401-49a9-bf7e-194827dff403 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:01.731073 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056c4e7-b401-49a9-bf7e-194827dff403 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:01.744146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2056c4e7-b401-49a9-bf7e-194827dff403 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:01.753124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-2056c4e7-b401-49a9-bf7e-194827dff403 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:51:01.753124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:01.753124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:01.753124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:01.753124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:01.753124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:01.753124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:01.753124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:01.753124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:01.753124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:01.753124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:01.753124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:01.753124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:01.753124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:01.753124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:01.753124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:01.754127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:01.754127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:01.754127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:01.754127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:01.754127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:01.754127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:01.754127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:01.754127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:01.754127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:01.754127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:01.754127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:01.754127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:01.754127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:01.754127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:01.754127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:01.754127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:01.755109 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:01.755109 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:01.755109 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:01.755109 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:01.755109 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:01.755109 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:01.755109 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:01.755109 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:01.755109 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:01.755109 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:01.755109 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:51:01.755109 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:01.755109 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 91/180] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:51:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:01.760804 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0eb624-8a64-4d36-8d2a-32647a54a70b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:01.760804 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0eb624-8a64-4d36-8d2a-32647a54a70b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:01.760804 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0eb624-8a64-4d36-8d2a-32647a54a70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:01.779185 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-9a0eb624-8a64-4d36-8d2a-32647a54a70b None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:01.780653 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9a0eb624-8a64-4d36-8d2a-32647a54a70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:01.804582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7194d74-58c5-4025-8c68-f8438a1f35f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:01.806090 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7194d74-58c5-4025-8c68-f8438a1f35f9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:01.851669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 92/181] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:01.852897 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0eb624-8a64-4d36-8d2a-32647a54a70b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:01.866146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:01.937660 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 90/182] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:51:01.939350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7194d74-58c5-4025-8c68-f8438a1f35f9 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:01.967675 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:01.979002 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0eb624-8a64-4d36-8d2a-32647a54a70b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:02.021535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87454e53-caa4-4cdc-a717-242cd8a76a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:02.023359 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87454e53-caa4-4cdc-a717-242cd8a76a1a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:02.024353 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87454e53-caa4-4cdc-a717-242cd8a76a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:02.024834 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87454e53-caa4-4cdc-a717-242cd8a76a1a None admin] PATH_INFO: `/v3/projects/ac5d363867c44c349f5be02f57cd5958` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:02.026299 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87454e53-caa4-4cdc-a717-242cd8a76a1a None admin] RBAC: Authorizing `identity:get_project(project_id=ac5d363867c44c349f5be02f57cd5958)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:02.028475 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87454e53-caa4-4cdc-a717-242cd8a76a1a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:02.031083 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 93/183] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:01 2022] GET /identity/v3/projects/ac5d363867c44c349f5be02f57cd5958 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:02.032421 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87454e53-caa4-4cdc-a717-242cd8a76a1a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:02.056754 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7194d74-58c5-4025-8c68-f8438a1f35f9 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:02.097750 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f1761-3e48-449f-b971-b7bc934b38dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:02.098366 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f1761-3e48-449f-b971-b7bc934b38dd None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:02.098646 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f1761-3e48-449f-b971-b7bc934b38dd None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:02.098920 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f1761-3e48-449f-b971-b7bc934b38dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:02.099774 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f1761-3e48-449f-b971-b7bc934b38dd None admin] RBAC: Adding query filter params (role.id=5d9045e45840408a92ceca71e9184ba2, scope.project.id=ac5d363867c44c349f5be02f57cd5958, user.id=938f59e04b4b4ad7bb6b69e44e819036) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:02.099981 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f1761-3e48-449f-b971-b7bc934b38dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:02.101424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f1761-3e48-449f-b971-b7bc934b38dd None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:02.107673 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 91/184] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:51:01 2022] GET /identity/v3/role_assignments?user.id=938f59e04b4b4ad7bb6b69e44e819036&scope.project.id=ac5d363867c44c349f5be02f57cd5958&role.id=5d9045e45840408a92ceca71e9184ba2 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:02.114500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f1761-3e48-449f-b971-b7bc934b38dd None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:02.152964 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:02.176759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87454e53-caa4-4cdc-a717-242cd8a76a1a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:02.218234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643867e6-d6e1-4134-93bc-b7e050e6f41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:02.219134 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643867e6-d6e1-4134-93bc-b7e050e6f41a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:02.219243 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643867e6-d6e1-4134-93bc-b7e050e6f41a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:02.219494 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643867e6-d6e1-4134-93bc-b7e050e6f41a None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:02.220209 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643867e6-d6e1-4134-93bc-b7e050e6f41a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:02.220494 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643867e6-d6e1-4134-93bc-b7e050e6f41a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:02.222436 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643867e6-d6e1-4134-93bc-b7e050e6f41a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:02.237504 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 94/185] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:02.237504 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643867e6-d6e1-4134-93bc-b7e050e6f41a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:02.263297 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f1761-3e48-449f-b971-b7bc934b38dd None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:02.268254 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:02.305098 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414d760-ca39-44ef-89da-7065dec4fbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:02.305754 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414d760-ca39-44ef-89da-7065dec4fbfa None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:02.306038 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414d760-ca39-44ef-89da-7065dec4fbfa None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:02.306312 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414d760-ca39-44ef-89da-7065dec4fbfa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:02.307440 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d414d760-ca39-44ef-89da-7065dec4fbfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:02.308929 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d414d760-ca39-44ef-89da-7065dec4fbfa None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:02.310515 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 92/186] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:02.311366 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414d760-ca39-44ef-89da-7065dec4fbfa None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:02.350857 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643867e6-d6e1-4134-93bc-b7e050e6f41a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:02.388384 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae32b0-7587-4a91-89d3-a2888435953a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:02.389025 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae32b0-7587-4a91-89d3-a2888435953a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:02.389312 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae32b0-7587-4a91-89d3-a2888435953a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:02.389587 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae32b0-7587-4a91-89d3-a2888435953a None admin] PATH_INFO: `/v3/projects/2aed3ce25de843aa9cdac9e9cd162d0b` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:02.390954 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae32b0-7587-4a91-89d3-a2888435953a None admin] RBAC: Authorizing `identity:get_project(project_id=2aed3ce25de843aa9cdac9e9cd162d0b)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:02.392467 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae32b0-7587-4a91-89d3-a2888435953a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:02.394719 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 95/187] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:02 2022] GET /identity/v3/projects/2aed3ce25de843aa9cdac9e9cd162d0b => generated 312 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:02.395487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae32b0-7587-4a91-89d3-a2888435953a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:02.405003 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:02.424035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414d760-ca39-44ef-89da-7065dec4fbfa None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:02.457629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319338f-b765-4620-bc5a-3be463cbe702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:02.458235 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319338f-b765-4620-bc5a-3be463cbe702 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:02.458513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319338f-b765-4620-bc5a-3be463cbe702 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:02.458789 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319338f-b765-4620-bc5a-3be463cbe702 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:02.459500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319338f-b765-4620-bc5a-3be463cbe702 None admin] RBAC: Adding query filter params (role.id=89a3bc954b0a45bfa40a16db0ad2e51c, scope.project.id=ac5d363867c44c349f5be02f57cd5958, user.id=938f59e04b4b4ad7bb6b69e44e819036) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:02.459841 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319338f-b765-4620-bc5a-3be463cbe702 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:02.461326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319338f-b765-4620-bc5a-3be463cbe702 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:02.466911 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 93/188] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:51:02 2022] GET /identity/v3/role_assignments?user.id=938f59e04b4b4ad7bb6b69e44e819036&scope.project.id=ac5d363867c44c349f5be02f57cd5958&role.id=89a3bc954b0a45bfa40a16db0ad2e51c => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:02.470124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94865b-b814-46d5-9782-ade96c6b2bf2 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:02.470124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94865b-b814-46d5-9782-ade96c6b2bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:02.470124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94865b-b814-46d5-9782-ade96c6b2bf2 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:02.470390 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 94/189] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:02.471019 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94865b-b814-46d5-9782-ade96c6b2bf2 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:02.502012 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:02.510877 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae32b0-7587-4a91-89d3-a2888435953a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:02.552097 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134343f7-13b5-4c29-bec8-cc089157cd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:02.552754 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134343f7-13b5-4c29-bec8-cc089157cd69 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:02.553036 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134343f7-13b5-4c29-bec8-cc089157cd69 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:02.553310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134343f7-13b5-4c29-bec8-cc089157cd69 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:02.565358 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134343f7-13b5-4c29-bec8-cc089157cd69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:02.570974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-134343f7-13b5-4c29-bec8-cc089157cd69 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:51:02.570974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:02.570974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:02.570974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:02.570974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:02.570974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:02.570974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:02.570974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:02.570974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:02.570974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:02.570974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:02.570974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:02.570974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:02.570974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:02.570974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:02.570974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:02.571951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:02.571951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:02.571951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:02.571951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:02.571951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:02.571951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:02.571951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:02.571951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:02.571951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:02.571951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:02.571951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:02.571951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:02.571951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:02.571951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:02.571951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:02.571951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:02.572939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:02.572939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:02.572939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:02.572939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:02.572939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:02.572939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:02.572939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:02.572939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:02.572939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:02.572939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:02.572939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:51:02.572939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:02.572939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 96/190] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 00:51:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:02.575576 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134343f7-13b5-4c29-bec8-cc089157cd69 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:02.600314 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94865b-b814-46d5-9782-ade96c6b2bf2 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:02.638974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352050f2-c9ac-4eae-b3cc-c61199530e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:02.639700 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352050f2-c9ac-4eae-b3cc-c61199530e38 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:02.639993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352050f2-c9ac-4eae-b3cc-c61199530e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:02.640546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352050f2-c9ac-4eae-b3cc-c61199530e38 None admin] PATH_INFO: `/v3/users/1c485f5f331144bf8d9c5ed6e804cc29` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:02.657941 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352050f2-c9ac-4eae-b3cc-c61199530e38 None admin] RBAC: Authorizing `identity:get_user(user_id=1c485f5f331144bf8d9c5ed6e804cc29)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:02.660200 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352050f2-c9ac-4eae-b3cc-c61199530e38 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:02.662356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 95/191] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:51:02 2022] GET /identity/v3/users/1c485f5f331144bf8d9c5ed6e804cc29 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:02.662731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352050f2-c9ac-4eae-b3cc-c61199530e38 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:02.666949 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:02.694475 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134343f7-13b5-4c29-bec8-cc089157cd69 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:02.737434 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4538c91-f14c-4ee9-aa9c-9e58b7a4aaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:02.738047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4538c91-f14c-4ee9-aa9c-9e58b7a4aaa8 None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:02.738325 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4538c91-f14c-4ee9-aa9c-9e58b7a4aaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:02.738596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4538c91-f14c-4ee9-aa9c-9e58b7a4aaa8 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:02.739256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4538c91-f14c-4ee9-aa9c-9e58b7a4aaa8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:02.740774 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4538c91-f14c-4ee9-aa9c-9e58b7a4aaa8 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:02.761669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 97/192] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 00:51:02 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:51:02.762706 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64950cf6-dfc4-4fce-98ec-b09de8313db8 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:02.762998 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64950cf6-dfc4-4fce-98ec-b09de8313db8 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:02.763275 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64950cf6-dfc4-4fce-98ec-b09de8313db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:02.782168 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352050f2-c9ac-4eae-b3cc-c61199530e38 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:02.782931 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-64950cf6-dfc4-4fce-98ec-b09de8313db8 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:02.784773 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-64950cf6-dfc4-4fce-98ec-b09de8313db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:02.788848 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:02.822989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c9c63-a16b-48a5-a822-87fe0fd7d008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:02.823653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c9c63-a16b-48a5-a822-87fe0fd7d008 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:02.823990 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c9c63-a16b-48a5-a822-87fe0fd7d008 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:02.824214 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c9c63-a16b-48a5-a822-87fe0fd7d008 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:02.825055 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c9c63-a16b-48a5-a822-87fe0fd7d008 None admin] RBAC: Adding query filter params (role.id=5d9045e45840408a92ceca71e9184ba2, scope.project.id=2aed3ce25de843aa9cdac9e9cd162d0b, user.id=938f59e04b4b4ad7bb6b69e44e819036) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:02.825284 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c9c63-a16b-48a5-a822-87fe0fd7d008 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:02.826797 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c9c63-a16b-48a5-a822-87fe0fd7d008 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:02.833764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 96/193] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:51:02 2022] GET /identity/v3/role_assignments?user.id=938f59e04b4b4ad7bb6b69e44e819036&scope.project.id=2aed3ce25de843aa9cdac9e9cd162d0b&role.id=5d9045e45840408a92ceca71e9184ba2 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:02.833764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c9c63-a16b-48a5-a822-87fe0fd7d008 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:02.878623 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 98/194] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:02.880091 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64950cf6-dfc4-4fce-98ec-b09de8313db8 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:02.929127 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:02.984341 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c9c63-a16b-48a5-a822-87fe0fd7d008 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:03.035981 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6de9a6-bdc6-4c84-9d3e-362ec61d29e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:03.037457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6de9a6-bdc6-4c84-9d3e-362ec61d29e1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:03.037457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6de9a6-bdc6-4c84-9d3e-362ec61d29e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:03.037457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6de9a6-bdc6-4c84-9d3e-362ec61d29e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:03.037877 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6de9a6-bdc6-4c84-9d3e-362ec61d29e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:03.040658 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6de9a6-bdc6-4c84-9d3e-362ec61d29e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:03.040658 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6de9a6-bdc6-4c84-9d3e-362ec61d29e1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:03.043358 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64950cf6-dfc4-4fce-98ec-b09de8313db8 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:03.054870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 97/195] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:51:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:03.061823 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6de9a6-bdc6-4c84-9d3e-362ec61d29e1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:03.109912 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059f3b0-73f5-434a-bf1a-6b0254f84825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:03.110979 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059f3b0-73f5-434a-bf1a-6b0254f84825 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:03.110979 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059f3b0-73f5-434a-bf1a-6b0254f84825 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:03.111149 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059f3b0-73f5-434a-bf1a-6b0254f84825 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:03.125562 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:03.203165 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c059f3b0-73f5-434a-bf1a-6b0254f84825 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:03.205021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c059f3b0-73f5-434a-bf1a-6b0254f84825 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:03.211768 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6de9a6-bdc6-4c84-9d3e-362ec61d29e1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:03.260201 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:03.268395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc983e-cadf-4f54-999f-5424d8856fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:03.269084 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc983e-cadf-4f54-999f-5424d8856fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:03.269471 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc983e-cadf-4f54-999f-5424d8856fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:03.269827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc983e-cadf-4f54-999f-5424d8856fe0 None admin] PATH_INFO: `/v3/users/1c485f5f331144bf8d9c5ed6e804cc29` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:03.271019 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dc983e-cadf-4f54-999f-5424d8856fe0 None admin] RBAC: Authorizing `identity:get_user(user_id=1c485f5f331144bf8d9c5ed6e804cc29)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:03.272552 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dc983e-cadf-4f54-999f-5424d8856fe0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:03.274165 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 98/196] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:51:03 2022] GET /identity/v3/users/1c485f5f331144bf8d9c5ed6e804cc29 => generated 283 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:03.276448 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc983e-cadf-4f54-999f-5424d8856fe0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:03.333279 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 99/197] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:51:03.339140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059f3b0-73f5-434a-bf1a-6b0254f84825 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:03.386942 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:03.398796 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc983e-cadf-4f54-999f-5424d8856fe0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:03.438343 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674a59e-5912-4fde-a4ad-89e05d23c6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:03.439238 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674a59e-5912-4fde-a4ad-89e05d23c6ed None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:03.440373 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674a59e-5912-4fde-a4ad-89e05d23c6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:03.440656 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674a59e-5912-4fde-a4ad-89e05d23c6ed None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:03.451962 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674a59e-5912-4fde-a4ad-89e05d23c6ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:03.458507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-b674a59e-5912-4fde-a4ad-89e05d23c6ed None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 00:51:03.458507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:03.458507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:03.458507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:03.458507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:03.458507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:03.458507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:03.458507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:03.458507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:03.458507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:03.458507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:03.458507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:03.458507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:03.458507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:03.458507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:03.458507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:03.459615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:03.459615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:03.459615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:03.459615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:03.459615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:03.459615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:03.459615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:03.459615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:03.459615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:03.459615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:03.459615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:03.459615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:03.459615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:03.459615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:03.459615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:03.459615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:03.460630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:03.460630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:03.460630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:03.460630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:03.460630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:03.460630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:03.460630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:03.460630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:03.460630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:03.460630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:03.460630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 00:51:03.460630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:03.462273 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 99/198] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:03.467425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674a59e-5912-4fde-a4ad-89e05d23c6ed None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:03.472712 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059f3b0-73f5-434a-bf1a-6b0254f84825 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:03.511391 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851cdac-5e1a-40de-9054-50bf73cfcd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:03.512328 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851cdac-5e1a-40de-9054-50bf73cfcd84 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:03.512810 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851cdac-5e1a-40de-9054-50bf73cfcd84 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:03.513279 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851cdac-5e1a-40de-9054-50bf73cfcd84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:03.528494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:03.575749 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674a59e-5912-4fde-a4ad-89e05d23c6ed None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:03.588248 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9851cdac-5e1a-40de-9054-50bf73cfcd84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:03.589991 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9851cdac-5e1a-40de-9054-50bf73cfcd84 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:03.613777 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acbc6a-d13e-487e-bb4a-b9b4236ce8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:03.614523 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acbc6a-d13e-487e-bb4a-b9b4236ce8e8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:03.614918 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acbc6a-d13e-487e-bb4a-b9b4236ce8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:03.615265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68acbc6a-d13e-487e-bb4a-b9b4236ce8e8 None admin] PATH_INFO: `/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:03.616510 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68acbc6a-d13e-487e-bb4a-b9b4236ce8e8 None admin] RBAC: Authorizing `identity:get_project(project_id=99c39c8f77a74048b8069e2cf0f83ae6)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:03.618087 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68acbc6a-d13e-487e-bb4a-b9b4236ce8e8 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:03.619961 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 100/199] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:03 2022] GET /identity/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:03.624942 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acbc6a-d13e-487e-bb4a-b9b4236ce8e8 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:03.647467 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:03.716447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 100/200] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:51:03.717352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851cdac-5e1a-40de-9054-50bf73cfcd84 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:03.767289 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68acbc6a-d13e-487e-bb4a-b9b4236ce8e8 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:03.795189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:03.806021 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7f9c2-356e-4fa1-bae4-9af912227729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:03.806903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7f9c2-356e-4fa1-bae4-9af912227729 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:03.807278 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7f9c2-356e-4fa1-bae4-9af912227729 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:03.807680 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7f9c2-356e-4fa1-bae4-9af912227729 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:03.808552 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc7f9c2-356e-4fa1-bae4-9af912227729 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:03.808962 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc7f9c2-356e-4fa1-bae4-9af912227729 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:03.810527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc7f9c2-356e-4fa1-bae4-9af912227729 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:03.824720 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 101/201] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:51:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:03.824720 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7f9c2-356e-4fa1-bae4-9af912227729 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:03.856956 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851cdac-5e1a-40de-9054-50bf73cfcd84 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:03.906439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa5cf2-8a5d-43a9-ab94-a35d4b051967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:03.907302 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa5cf2-8a5d-43a9-ab94-a35d4b051967 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:03.907756 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa5cf2-8a5d-43a9-ab94-a35d4b051967 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:03.908224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa5cf2-8a5d-43a9-ab94-a35d4b051967 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:03.909312 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa5cf2-8a5d-43a9-ab94-a35d4b051967 None admin] RBAC: Adding query filter params (role.id=ada268ea63354a0297070369dfe3c6cc, scope.project.id=99c39c8f77a74048b8069e2cf0f83ae6, user.id=1c485f5f331144bf8d9c5ed6e804cc29) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:03.909772 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa5cf2-8a5d-43a9-ab94-a35d4b051967 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:03.911942 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa5cf2-8a5d-43a9-ab94-a35d4b051967 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:03.927117 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 101/202] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:51:03 2022] GET /identity/v3/role_assignments?user.id=1c485f5f331144bf8d9c5ed6e804cc29&scope.project.id=99c39c8f77a74048b8069e2cf0f83ae6&role.id=ada268ea63354a0297070369dfe3c6cc => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:03.927117 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa5cf2-8a5d-43a9-ab94-a35d4b051967 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:03.934926 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:03.960589 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7f9c2-356e-4fa1-bae4-9af912227729 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:04.030164 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d9438-107c-439c-8752-15c6f157564a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:04.030741 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d9438-107c-439c-8752-15c6f157564a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:04.031018 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d9438-107c-439c-8752-15c6f157564a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:04.031292 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d9438-107c-439c-8752-15c6f157564a None admin] PATH_INFO: `/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:04.037571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d9438-107c-439c-8752-15c6f157564a None admin] RBAC: Authorizing `identity:get_project(project_id=99c39c8f77a74048b8069e2cf0f83ae6)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:04.039413 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d9438-107c-439c-8752-15c6f157564a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:04.044957 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 102/203] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:03 2022] GET /identity/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:04.048576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d9438-107c-439c-8752-15c6f157564a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:04.134796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa5cf2-8a5d-43a9-ab94-a35d4b051967 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:04.175286 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:04.188024 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bace56-05d4-48a5-be01-b7329c182d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:04.188024 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bace56-05d4-48a5-be01-b7329c182d76 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:04.188024 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bace56-05d4-48a5-be01-b7329c182d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:04.188297 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bace56-05d4-48a5-be01-b7329c182d76 None admin] PATH_INFO: `/v3/users/7228f12915de43ab88e029a4d42ebd93` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:04.190945 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bace56-05d4-48a5-be01-b7329c182d76 None admin] RBAC: Authorizing `identity:get_user(user_id=7228f12915de43ab88e029a4d42ebd93)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:04.193730 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bace56-05d4-48a5-be01-b7329c182d76 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:04.195665 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 102/204] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:51:03 2022] GET /identity/v3/users/7228f12915de43ab88e029a4d42ebd93 => generated 289 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:04.201313 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bace56-05d4-48a5-be01-b7329c182d76 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:04.235256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d9438-107c-439c-8752-15c6f157564a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:04.278008 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5e20c-506b-404a-adec-5e21b9d2620f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:04.279445 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5e20c-506b-404a-adec-5e21b9d2620f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:04.280105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5e20c-506b-404a-adec-5e21b9d2620f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:04.282368 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5e20c-506b-404a-adec-5e21b9d2620f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:04.283864 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a5e20c-506b-404a-adec-5e21b9d2620f None admin] RBAC: Adding query filter params (role.id=89a3bc954b0a45bfa40a16db0ad2e51c, scope.project.id=99c39c8f77a74048b8069e2cf0f83ae6, user.id=1c485f5f331144bf8d9c5ed6e804cc29) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:04.285642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a5e20c-506b-404a-adec-5e21b9d2620f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:04.287309 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a5e20c-506b-404a-adec-5e21b9d2620f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:04.300464 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 103/205] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:51:04 2022] GET /identity/v3/role_assignments?user.id=1c485f5f331144bf8d9c5ed6e804cc29&scope.project.id=99c39c8f77a74048b8069e2cf0f83ae6&role.id=89a3bc954b0a45bfa40a16db0ad2e51c => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:04.320763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:04.348298 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bace56-05d4-48a5-be01-b7329c182d76 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:04.387584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226bf331-f382-4e2e-bbeb-61b77ff8938f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:04.388214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bf331-f382-4e2e-bbeb-61b77ff8938f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:04.388495 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bf331-f382-4e2e-bbeb-61b77ff8938f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:04.388770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bf331-f382-4e2e-bbeb-61b77ff8938f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:04.473776 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226bf331-f382-4e2e-bbeb-61b77ff8938f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:04.475291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226bf331-f382-4e2e-bbeb-61b77ff8938f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:04.584642 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:04.627243 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 103/206] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:51:04.633685 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5e20c-506b-404a-adec-5e21b9d2620f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:04.758039 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5e20c-506b-404a-adec-5e21b9d2620f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:04.779477 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:04.794227 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f53321-b24b-4251-9ec4-7d9d5a3cb3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:04.794863 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f53321-b24b-4251-9ec4-7d9d5a3cb3c2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:04.795144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f53321-b24b-4251-9ec4-7d9d5a3cb3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:04.795417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f53321-b24b-4251-9ec4-7d9d5a3cb3c2 None admin] PATH_INFO: `/v3/projects/ac5d363867c44c349f5be02f57cd5958` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:04.796573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f53321-b24b-4251-9ec4-7d9d5a3cb3c2 None admin] RBAC: Authorizing `identity:get_project(project_id=ac5d363867c44c349f5be02f57cd5958)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:04.798023 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f53321-b24b-4251-9ec4-7d9d5a3cb3c2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:04.800451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 104/207] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:04 2022] GET /identity/v3/projects/ac5d363867c44c349f5be02f57cd5958 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:04.806362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226bf331-f382-4e2e-bbeb-61b77ff8938f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:04.926409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226bf331-f382-4e2e-bbeb-61b77ff8938f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:04.962770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea85db-df36-4714-af3b-341f1ba4e14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:04.963395 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea85db-df36-4714-af3b-341f1ba4e14b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:04.963708 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea85db-df36-4714-af3b-341f1ba4e14b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:04.963986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea85db-df36-4714-af3b-341f1ba4e14b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:04.964715 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea85db-df36-4714-af3b-341f1ba4e14b None admin] RBAC: Adding query filter params (role.id=f330ff772cca47c4b89a406784e00800, scope.project.id=ac5d363867c44c349f5be02f57cd5958, user.id=7228f12915de43ab88e029a4d42ebd93) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:04.965087 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea85db-df36-4714-af3b-341f1ba4e14b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:04.966553 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea85db-df36-4714-af3b-341f1ba4e14b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:04.974208 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 104/208] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:51:04 2022] GET /identity/v3/role_assignments?user.id=7228f12915de43ab88e029a4d42ebd93&scope.project.id=ac5d363867c44c349f5be02f57cd5958&role.id=f330ff772cca47c4b89a406784e00800 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:05.422880 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620fa90-52f1-45e8-8107-faec1c4f120c None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:05.423393 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620fa90-52f1-45e8-8107-faec1c4f120c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:05.423498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620fa90-52f1-45e8-8107-faec1c4f120c None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:05.425512 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 105/209] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:05.431299 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f783e8-1f0c-46c2-9fd5-745d786a7244 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:05.431781 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f783e8-1f0c-46c2-9fd5-745d786a7244 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:05.431856 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f783e8-1f0c-46c2-9fd5-745d786a7244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:05.454715 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-92f783e8-1f0c-46c2-9fd5-745d786a7244 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:05.456388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-92f783e8-1f0c-46c2-9fd5-745d786a7244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:05.496339 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:05.550575 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 105/210] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:05.722879 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6414c-ff7b-4d43-a1d9-9006a0bb9a7a None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:05.723202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6414c-ff7b-4d43-a1d9-9006a0bb9a7a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:05.723363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6414c-ff7b-4d43-a1d9-9006a0bb9a7a None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:05.724769 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 106/211] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:05.730071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c67110-5902-40d4-889a-48a891f07c53 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:05.730586 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c67110-5902-40d4-889a-48a891f07c53 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:05.731080 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c67110-5902-40d4-889a-48a891f07c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:05.749482 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-58c67110-5902-40d4-889a-48a891f07c53 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:05.750924 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-58c67110-5902-40d4-889a-48a891f07c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:05.820207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 106/212] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:05.827580 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6414c-ff7b-4d43-a1d9-9006a0bb9a7a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:05.928077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6414c-ff7b-4d43-a1d9-9006a0bb9a7a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:05.963894 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dddc0-1d7e-4730-b9ce-16d0d14becbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:05.964597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dddc0-1d7e-4730-b9ce-16d0d14becbd None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:05.964875 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dddc0-1d7e-4730-b9ce-16d0d14becbd None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:05.965151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dddc0-1d7e-4730-b9ce-16d0d14becbd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:05.976886 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482dddc0-1d7e-4730-b9ce-16d0d14becbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:05.981577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-482dddc0-1d7e-4730-b9ce-16d0d14becbd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:05.981577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:05.981577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:05.981577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:05.981577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:05.981577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:05.981577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:05.981577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:05.981577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:05.981577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:05.981577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:05.981577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:05.981577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:05.981577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:05.981577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:05.981577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:05.982460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:05.982460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:05.982460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:05.982460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:05.982460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:05.982460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:05.982460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:05.982460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:05.982460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:05.982460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:05.982460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:05.982460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:05.982460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:05.982460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:05.982460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:05.982460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:05.983256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:05.983256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:05.983256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:05.983256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:05.983256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:05.983256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:05.983256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:05.983256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:05.983256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:05.983256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:05.983256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:05.983256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:05.983256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 107/213] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:05.989512 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c67110-5902-40d4-889a-48a891f07c53 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:06.089917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:06.096553 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c67110-5902-40d4-889a-48a891f07c53 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:06.109386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dddc0-1d7e-4730-b9ce-16d0d14becbd None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:06.136728 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf620b-744e-4b83-91be-658d2758964d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:06.137351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf620b-744e-4b83-91be-658d2758964d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:06.137632 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf620b-744e-4b83-91be-658d2758964d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:06.137908 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf620b-744e-4b83-91be-658d2758964d None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:06.138699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf620b-744e-4b83-91be-658d2758964d None admin] RBAC: Adding query filter params (name=member) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:06.139016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf620b-744e-4b83-91be-658d2758964d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:06.140713 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bf620b-744e-4b83-91be-658d2758964d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:06.153824 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 107/214] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:51:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:06.161822 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf620b-744e-4b83-91be-658d2758964d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:06.203166 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:06.299307 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dddc0-1d7e-4730-b9ce-16d0d14becbd None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:06.331075 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf620b-744e-4b83-91be-658d2758964d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:06.390443 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f85311a-72a3-42ce-a7ed-e289063ad457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:06.392039 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85311a-72a3-42ce-a7ed-e289063ad457 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:06.393119 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85311a-72a3-42ce-a7ed-e289063ad457 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:06.394208 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85311a-72a3-42ce-a7ed-e289063ad457 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:06.396454 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f85311a-72a3-42ce-a7ed-e289063ad457 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:06.399030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4c80b-671c-4b23-88c6-596b6705db8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:06.400194 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4c80b-671c-4b23-88c6-596b6705db8a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:06.402301 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f85311a-72a3-42ce-a7ed-e289063ad457 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:06.402795 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4c80b-671c-4b23-88c6-596b6705db8a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:06.403273 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4c80b-671c-4b23-88c6-596b6705db8a None admin] PATH_INFO: `/v3/users/484f60d655cd421c927b28176787975d` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:06.411388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 108/215] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:06.412601 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f85311a-72a3-42ce-a7ed-e289063ad457 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:06.443219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a4c80b-671c-4b23-88c6-596b6705db8a None admin] RBAC: Authorizing `identity:get_user(user_id=484f60d655cd421c927b28176787975d)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:06.444759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a4c80b-671c-4b23-88c6-596b6705db8a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:06.446840 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 108/216] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:51:06 2022] GET /identity/v3/users/484f60d655cd421c927b28176787975d => generated 297 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:06.451867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4c80b-671c-4b23-88c6-596b6705db8a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:06.482689 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:06.557471 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f85311a-72a3-42ce-a7ed-e289063ad457 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:06.583658 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:06.600025 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4c80b-671c-4b23-88c6-596b6705db8a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:06.606592 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d5e72-d065-4de0-a903-ce47fec9701a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:06.607355 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d5e72-d065-4de0-a903-ce47fec9701a None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:06.607780 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d5e72-d065-4de0-a903-ce47fec9701a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:06.608157 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d5e72-d065-4de0-a903-ce47fec9701a None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:06.608931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d5e72-d065-4de0-a903-ce47fec9701a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:06.614811 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d5e72-d065-4de0-a903-ce47fec9701a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:06.641067 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3855c6a9-6a06-474e-9053-badfbfe406a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:06.641807 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 109/217] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 00:51:06 2022] POST /identity/v3/users => generated 297 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:51:06.641917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855c6a9-6a06-474e-9053-badfbfe406a1 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:06.641991 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855c6a9-6a06-474e-9053-badfbfe406a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:06.642203 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855c6a9-6a06-474e-9053-badfbfe406a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:06.645020 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074b4fd-0bc8-408e-9143-e6850a6e073b None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:06.645588 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074b4fd-0bc8-408e-9143-e6850a6e073b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:06.646138 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074b4fd-0bc8-408e-9143-e6850a6e073b None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:06.647917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 110/218] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:06.663585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31843ea4-ab2d-4bbe-8997-cd35aecd1a2d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:06.663585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31843ea4-ab2d-4bbe-8997-cd35aecd1a2d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:06.663585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31843ea4-ab2d-4bbe-8997-cd35aecd1a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:06.674856 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-31843ea4-ab2d-4bbe-8997-cd35aecd1a2d None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:06.676116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-31843ea4-ab2d-4bbe-8997-cd35aecd1a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:06.768772 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3855c6a9-6a06-474e-9053-badfbfe406a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:06.770338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3855c6a9-6a06-474e-9053-badfbfe406a1 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:06.777608 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 111/219] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:51:06.798798 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:06.892923 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 109/220] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:51:06.898442 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31843ea4-ab2d-4bbe-8997-cd35aecd1a2d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:06.998699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957eca30-7693-48c8-b94c-d18ccf437848 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:06.999147 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957eca30-7693-48c8-b94c-d18ccf437848 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:06.999529 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957eca30-7693-48c8-b94c-d18ccf437848 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:07.001295 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 110/221] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:51:07.010247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5d438-5993-4676-b794-2d4d41fb1323 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:07.010529 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5d438-5993-4676-b794-2d4d41fb1323 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:07.010811 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5d438-5993-4676-b794-2d4d41fb1323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:07.011364 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31843ea4-ab2d-4bbe-8997-cd35aecd1a2d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:07.013189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:07.038521 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-0ef5d438-5993-4676-b794-2d4d41fb1323 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:07.039844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0ef5d438-5993-4676-b794-2d4d41fb1323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:07.060183 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fbae5-740a-4761-b246-64b990f7fb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:07.060843 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fbae5-740a-4761-b246-64b990f7fb58 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:07.061143 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fbae5-740a-4761-b246-64b990f7fb58 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:07.061399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fbae5-740a-4761-b246-64b990f7fb58 None admin] PATH_INFO: `/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:07.063338 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606fbae5-740a-4761-b246-64b990f7fb58 None admin] RBAC: Authorizing `identity:get_project(project_id=99c39c8f77a74048b8069e2cf0f83ae6)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:07.065538 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606fbae5-740a-4761-b246-64b990f7fb58 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:07.067472 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 112/222] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:06 2022] GET /identity/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:07.068281 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fbae5-740a-4761-b246-64b990f7fb58 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:07.130433 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 111/223] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:51:07.131194 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5d438-5993-4676-b794-2d4d41fb1323 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:07.213349 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:07.232822 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fbae5-740a-4761-b246-64b990f7fb58 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:07.272773 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5d438-5993-4676-b794-2d4d41fb1323 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:07.302942 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8570c3-bc67-459b-8b5c-e5f14005e791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:07.303600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8570c3-bc67-459b-8b5c-e5f14005e791 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:07.303883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8570c3-bc67-459b-8b5c-e5f14005e791 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:07.304154 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8570c3-bc67-459b-8b5c-e5f14005e791 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:07.326500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8570c3-bc67-459b-8b5c-e5f14005e791 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:07.332651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-9a8570c3-bc67-459b-8b5c-e5f14005e791 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:51:07.332651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:07.332651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:07.332651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:07.332651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:07.332651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:07.332651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:07.332651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:07.332651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:07.332651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:07.332651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:07.332651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:07.332651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:07.332651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:07.332651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:07.332651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:07.333546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:07.333546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:07.333546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:07.333546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:07.333546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:07.333546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:07.333546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:07.333546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:07.333546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:07.333546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:07.333546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:07.333546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:07.333546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:07.333546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:07.333546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:07.333546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:07.334325 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:07.334325 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:07.334325 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:07.334325 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:07.334325 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:07.334325 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:07.334325 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:07.334325 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:07.334325 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:07.334325 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:07.334325 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:51:07.334325 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:07.339970 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 113/224] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:51:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:07.345764 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886af2da-177b-4085-b896-dade8df0dc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:07.346424 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886af2da-177b-4085-b896-dade8df0dc9b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:07.346705 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886af2da-177b-4085-b896-dade8df0dc9b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:07.346839 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8570c3-bc67-459b-8b5c-e5f14005e791 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:07.347015 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886af2da-177b-4085-b896-dade8df0dc9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:07.347844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886af2da-177b-4085-b896-dade8df0dc9b None admin] RBAC: Adding query filter params (role.id=5d9045e45840408a92ceca71e9184ba2, scope.project.id=99c39c8f77a74048b8069e2cf0f83ae6, user.id=484f60d655cd421c927b28176787975d) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:07.348150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886af2da-177b-4085-b896-dade8df0dc9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:07.349591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886af2da-177b-4085-b896-dade8df0dc9b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:07.365573 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 112/225] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:51:07 2022] GET /identity/v3/role_assignments?user.id=484f60d655cd421c927b28176787975d&scope.project.id=99c39c8f77a74048b8069e2cf0f83ae6&role.id=5d9045e45840408a92ceca71e9184ba2 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:07.385831 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:07.456100 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886af2da-177b-4085-b896-dade8df0dc9b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:07.473557 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8570c3-bc67-459b-8b5c-e5f14005e791 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:07.555367 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5a8af-0c3c-42fc-930d-cdf0fb0673b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:07.557464 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5a8af-0c3c-42fc-930d-cdf0fb0673b6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:07.558162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5a8af-0c3c-42fc-930d-cdf0fb0673b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:07.559822 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5a8af-0c3c-42fc-930d-cdf0fb0673b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:07.560733 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5a8af-0c3c-42fc-930d-cdf0fb0673b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:07.561464 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5a8af-0c3c-42fc-930d-cdf0fb0673b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:07.563733 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5a8af-0c3c-42fc-930d-cdf0fb0673b6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:07.587384 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 114/226] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:07.593330 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5a8af-0c3c-42fc-930d-cdf0fb0673b6 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:07.641042 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886af2da-177b-4085-b896-dade8df0dc9b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:07.653265 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:07.679608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89269637-770b-45c5-b340-0175922b3a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:07.680224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89269637-770b-45c5-b340-0175922b3a35 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:07.680529 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89269637-770b-45c5-b340-0175922b3a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:07.680808 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89269637-770b-45c5-b340-0175922b3a35 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:07.693074 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89269637-770b-45c5-b340-0175922b3a35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:07.699200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-89269637-770b-45c5-b340-0175922b3a35 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:51:07.699200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:07.699200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:07.699200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:07.699200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:07.699200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:07.699200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:07.699200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:07.699200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:07.699200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:07.699200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:07.699200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:07.699200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:07.699200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:07.699200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:07.699200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:07.700138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:07.700138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:07.700138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:07.700138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:07.700138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:07.700138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:07.700138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:07.700138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:07.700138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:07.700138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:07.700138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:07.700138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:07.700138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:07.700138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:07.700138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:07.700138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:07.705377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:07.705377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:07.705377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:07.705377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:07.705377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:07.705377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:07.705377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:07.705377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:07.705377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:07.705377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:07.705377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:51:07.705377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:07.705377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 113/227] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 00:51:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:51:07.708498 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89269637-770b-45c5-b340-0175922b3a35 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:07.728365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:07.754343 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5a8af-0c3c-42fc-930d-cdf0fb0673b6 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:07.826480 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6d20b-fa0f-4139-bbdc-d9edad7dd441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:07.827127 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6d20b-fa0f-4139-bbdc-d9edad7dd441 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:07.827406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6d20b-fa0f-4139-bbdc-d9edad7dd441 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:07.827715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6d20b-fa0f-4139-bbdc-d9edad7dd441 None admin] PATH_INFO: `/v3/users/1c485f5f331144bf8d9c5ed6e804cc29` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:07.829774 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6d20b-fa0f-4139-bbdc-d9edad7dd441 None admin] RBAC: Authorizing `identity:get_user(user_id=1c485f5f331144bf8d9c5ed6e804cc29)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:07.831527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6d20b-fa0f-4139-bbdc-d9edad7dd441 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:07.833169 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 115/228] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:51:07 2022] GET /identity/v3/users/1c485f5f331144bf8d9c5ed6e804cc29 => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:07.840419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6d20b-fa0f-4139-bbdc-d9edad7dd441 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:07.901464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89269637-770b-45c5-b340-0175922b3a35 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:07.992941 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6d20b-fa0f-4139-bbdc-d9edad7dd441 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:08.012287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fa1c1-0946-48e7-bbbb-6ee6e594e60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:08.012925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fa1c1-0946-48e7-bbbb-6ee6e594e60e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:08.013213 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fa1c1-0946-48e7-bbbb-6ee6e594e60e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:08.013491 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fa1c1-0946-48e7-bbbb-6ee6e594e60e None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:08.014171 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425fa1c1-0946-48e7-bbbb-6ee6e594e60e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:08.014479 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425fa1c1-0946-48e7-bbbb-6ee6e594e60e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:08.016199 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425fa1c1-0946-48e7-bbbb-6ee6e594e60e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:08.027649 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065e135-254f-4454-87f9-fa1d351a5792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:08.028289 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065e135-254f-4454-87f9-fa1d351a5792 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:08.028570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065e135-254f-4454-87f9-fa1d351a5792 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:08.028844 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065e135-254f-4454-87f9-fa1d351a5792 None admin] PATH_INFO: `/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:08.029976 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065e135-254f-4454-87f9-fa1d351a5792 None admin] RBAC: Authorizing `identity:get_project(project_id=99c39c8f77a74048b8069e2cf0f83ae6)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:08.031267 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 114/229] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:51:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:08.031535 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065e135-254f-4454-87f9-fa1d351a5792 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:08.033708 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 116/230] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:07 2022] GET /identity/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:08.038398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fa1c1-0946-48e7-bbbb-6ee6e594e60e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:08.052042 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065e135-254f-4454-87f9-fa1d351a5792 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:08.089083 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:08.139918 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:08.170804 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fa1c1-0946-48e7-bbbb-6ee6e594e60e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:08.233989 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f563d52-6768-462c-89aa-2b176a1f1eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:08.234618 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f563d52-6768-462c-89aa-2b176a1f1eda None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:08.234899 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f563d52-6768-462c-89aa-2b176a1f1eda None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:08.235173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f563d52-6768-462c-89aa-2b176a1f1eda None admin] PATH_INFO: `/v3/users/1c485f5f331144bf8d9c5ed6e804cc29` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:08.237758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f563d52-6768-462c-89aa-2b176a1f1eda None admin] RBAC: Authorizing `identity:get_user(user_id=1c485f5f331144bf8d9c5ed6e804cc29)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:08.239249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f563d52-6768-462c-89aa-2b176a1f1eda None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:08.242656 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 115/231] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:51:08 2022] GET /identity/v3/users/1c485f5f331144bf8d9c5ed6e804cc29 => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:08.252833 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f563d52-6768-462c-89aa-2b176a1f1eda None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:08.263318 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065e135-254f-4454-87f9-fa1d351a5792 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:08.299596 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566640e8-8529-4123-85db-6fb65b44c44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:08.300716 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566640e8-8529-4123-85db-6fb65b44c44d None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:08.301219 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566640e8-8529-4123-85db-6fb65b44c44d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:08.301691 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566640e8-8529-4123-85db-6fb65b44c44d None admin] PATH_INFO: `/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6/users/1c485f5f331144bf8d9c5ed6e804cc29/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:08.304048 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566640e8-8529-4123-85db-6fb65b44c44d None admin] RBAC: Authorizing `identity:create_grant(project_id=99c39c8f77a74048b8069e2cf0f83ae6, user_id=1c485f5f331144bf8d9c5ed6e804cc29, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:08.305814 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566640e8-8529-4123-85db-6fb65b44c44d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:08.326117 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 117/232] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 00:51:08 2022] PUT /identity/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6/users/1c485f5f331144bf8d9c5ed6e804cc29/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:51:08.401559 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:08.407693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f563d52-6768-462c-89aa-2b176a1f1eda None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:08.457322 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bab955-f9d7-4a9c-bf23-fc3b864f6b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:08.458777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bab955-f9d7-4a9c-bf23-fc3b864f6b45 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:08.459287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bab955-f9d7-4a9c-bf23-fc3b864f6b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:08.459820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bab955-f9d7-4a9c-bf23-fc3b864f6b45 None admin] PATH_INFO: `/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:08.461282 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bab955-f9d7-4a9c-bf23-fc3b864f6b45 None admin] RBAC: Authorizing `identity:get_project(project_id=99c39c8f77a74048b8069e2cf0f83ae6)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:08.462996 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bab955-f9d7-4a9c-bf23-fc3b864f6b45 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:08.469902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 116/233] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:08 2022] GET /identity/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:08.472223 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566640e8-8529-4123-85db-6fb65b44c44d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:08.575927 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:08.593278 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566640e8-8529-4123-85db-6fb65b44c44d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:08.637850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005a68a-3b0e-41b8-889b-238b1978f04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:08.638519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005a68a-3b0e-41b8-889b-238b1978f04e None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:08.638856 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005a68a-3b0e-41b8-889b-238b1978f04e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:08.639121 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005a68a-3b0e-41b8-889b-238b1978f04e None admin] PATH_INFO: `/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6/users/1c485f5f331144bf8d9c5ed6e804cc29/roles/89a3bc954b0a45bfa40a16db0ad2e51c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:08.641711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2005a68a-3b0e-41b8-889b-238b1978f04e None admin] RBAC: Authorizing `identity:create_grant(project_id=99c39c8f77a74048b8069e2cf0f83ae6, user_id=1c485f5f331144bf8d9c5ed6e804cc29, role_id=89a3bc954b0a45bfa40a16db0ad2e51c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:08.643303 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2005a68a-3b0e-41b8-889b-238b1978f04e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:08.663291 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 118/234] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 00:51:08 2022] PUT /identity/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6/users/1c485f5f331144bf8d9c5ed6e804cc29/roles/89a3bc954b0a45bfa40a16db0ad2e51c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:51:09.440097 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdcea84-0b5a-4510-aa18-eb86f8d92637 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:09.440854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdcea84-0b5a-4510-aa18-eb86f8d92637 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:09.441363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdcea84-0b5a-4510-aa18-eb86f8d92637 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:09.443334 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 117/235] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:09.448977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1c63c-6e1c-4aa9-b1b2-486a4ae2cd1d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:09.449540 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1c63c-6e1c-4aa9-b1b2-486a4ae2cd1d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:09.450020 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f1c63c-6e1c-4aa9-b1b2-486a4ae2cd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:09.482049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d2f1c63c-6e1c-4aa9-b1b2-486a4ae2cd1d None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:09.483507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d2f1c63c-6e1c-4aa9-b1b2-486a4ae2cd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:09.499752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5a67d-1f73-479c-9cad-de4366e41c43 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:09.500767 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5a67d-1f73-479c-9cad-de4366e41c43 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:09.501246 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5a67d-1f73-479c-9cad-de4366e41c43 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:09.503368 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 118/236] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:09.508888 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ff1c6-e33b-429f-b0b0-ea41eef61852 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:09.509173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ff1c6-e33b-429f-b0b0-ea41eef61852 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:09.509450 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ff1c6-e33b-429f-b0b0-ea41eef61852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:09.544804 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-631ff1c6-e33b-429f-b0b0-ea41eef61852 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:09.545997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-631ff1c6-e33b-429f-b0b0-ea41eef61852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:09.593938 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 119/237] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:09.673585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 119/238] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:09.965721 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1c63c-6e1c-4aa9-b1b2-486a4ae2cd1d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:09.978071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ff1c6-e33b-429f-b0b0-ea41eef61852 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:10.131944 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:10.157410 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f1c63c-6e1c-4aa9-b1b2-486a4ae2cd1d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:10.166024 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:10.174635 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ff1c6-e33b-429f-b0b0-ea41eef61852 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:10.198863 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226e76e-834a-46ad-a41a-64406222cbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:10.199505 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226e76e-834a-46ad-a41a-64406222cbb4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:10.199814 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226e76e-834a-46ad-a41a-64406222cbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:10.200088 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226e76e-834a-46ad-a41a-64406222cbb4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:10.201412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226e76e-834a-46ad-a41a-64406222cbb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:10.203223 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226e76e-834a-46ad-a41a-64406222cbb4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:10.205767 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 120/239] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:10.212595 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226e76e-834a-46ad-a41a-64406222cbb4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:10.242579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9535a787-7bac-49d8-b971-82e5ae870fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:10.243146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535a787-7bac-49d8-b971-82e5ae870fdc None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:10.243430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535a787-7bac-49d8-b971-82e5ae870fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:10.243752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535a787-7bac-49d8-b971-82e5ae870fdc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:10.255632 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9535a787-7bac-49d8-b971-82e5ae870fdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:10.261062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-9535a787-7bac-49d8-b971-82e5ae870fdc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 00:51:10.261062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:10.261062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:10.261062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:10.261062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:10.261062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:10.261062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:10.261062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:10.261062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:10.261062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:10.261062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:10.261062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:10.261062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:10.261062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:10.261062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:10.261062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:10.261949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:10.261949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:10.261949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:10.261949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:10.261949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:10.261949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:10.261949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:10.261949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:10.261949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:10.261949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:10.261949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:10.261949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:10.261949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:10.261949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:10.261949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:10.261949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:10.262752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:10.262752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:10.262752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:10.262752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:10.262752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:10.262752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:10.262752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:10.262752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:10.262752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:10.262752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:10.262752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 00:51:10.262752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:10.262752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 120/240] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:51:10.263420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ac700-4ab1-43ca-9ae4-8279ad49e191 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:10.264026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ac700-4ab1-43ca-9ae4-8279ad49e191 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:10.264026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ac700-4ab1-43ca-9ae4-8279ad49e191 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:10.265505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 121/241] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:10.270154 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320ac700-4ab1-43ca-9ae4-8279ad49e191 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:10.341101 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226e76e-834a-46ad-a41a-64406222cbb4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:10.385115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad2f27-d1be-4047-abbd-f4b3ef8febf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:10.385729 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad2f27-d1be-4047-abbd-f4b3ef8febf0 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:10.386066 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad2f27-d1be-4047-abbd-f4b3ef8febf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:10.386283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad2f27-d1be-4047-abbd-f4b3ef8febf0 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:10.386940 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad2f27-d1be-4047-abbd-f4b3ef8febf0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:10.387296 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320ac700-4ab1-43ca-9ae4-8279ad49e191 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:10.388447 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad2f27-d1be-4047-abbd-f4b3ef8febf0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:10.417922 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 121/242] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 00:51:10 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:51:10.419636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cc724-fc26-4ed6-8f29-4f0d41f142ac None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:10.420048 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cc724-fc26-4ed6-8f29-4f0d41f142ac None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:10.420328 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cc724-fc26-4ed6-8f29-4f0d41f142ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:10.427735 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:10.428303 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82090ddd-f3ef-4e2d-98c0-3856c202cd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:10.429470 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82090ddd-f3ef-4e2d-98c0-3856c202cd8c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:10.429841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82090ddd-f3ef-4e2d-98c0-3856c202cd8c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:10.430345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82090ddd-f3ef-4e2d-98c0-3856c202cd8c None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:10.431182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82090ddd-f3ef-4e2d-98c0-3856c202cd8c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:10.431724 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82090ddd-f3ef-4e2d-98c0-3856c202cd8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:10.434379 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82090ddd-f3ef-4e2d-98c0-3856c202cd8c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:10.445425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a97cc724-fc26-4ed6-8f29-4f0d41f142ac None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:10.446144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 122/243] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:51:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:10.446645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a97cc724-fc26-4ed6-8f29-4f0d41f142ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:10.452548 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82090ddd-f3ef-4e2d-98c0-3856c202cd8c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:10.500303 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:10.584895 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 122/244] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:10.635369 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82090ddd-f3ef-4e2d-98c0-3856c202cd8c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:10.671935 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a8e1b-efe0-4a8e-a4b0-97a5f794ca2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:10.672577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a8e1b-efe0-4a8e-a4b0-97a5f794ca2c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:10.672859 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a8e1b-efe0-4a8e-a4b0-97a5f794ca2c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:10.673190 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a8e1b-efe0-4a8e-a4b0-97a5f794ca2c None admin] PATH_INFO: `/v3/users/630e0cb2a8a044b48e65e21a6aa81418` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:10.687603 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a8e1b-efe0-4a8e-a4b0-97a5f794ca2c None admin] RBAC: Authorizing `identity:get_user(user_id=630e0cb2a8a044b48e65e21a6aa81418)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:10.689108 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a8e1b-efe0-4a8e-a4b0-97a5f794ca2c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:10.692098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 123/245] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:51:10 2022] GET /identity/v3/users/630e0cb2a8a044b48e65e21a6aa81418 => generated 297 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:10.697656 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97cc724-fc26-4ed6-8f29-4f0d41f142ac None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:10.807333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97cc724-fc26-4ed6-8f29-4f0d41f142ac None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:10.838540 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:10.845422 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b486c4-343a-49bf-b4dd-36661469b0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:10.846778 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b486c4-343a-49bf-b4dd-36661469b0e6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:10.847278 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b486c4-343a-49bf-b4dd-36661469b0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:10.847577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b486c4-343a-49bf-b4dd-36661469b0e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:10.921612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b486c4-343a-49bf-b4dd-36661469b0e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:10.923184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b486c4-343a-49bf-b4dd-36661469b0e6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:10.936182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a8e1b-efe0-4a8e-a4b0-97a5f794ca2c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:11.046137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:11.064042 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 123/246] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:51:11.069767 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4c8a8-60f4-411f-9756-7eb360a05f02 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:11.070339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4c8a8-60f4-411f-9756-7eb360a05f02 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:11.070820 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4c8a8-60f4-411f-9756-7eb360a05f02 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:11.072634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 124/247] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:11.073777 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4c8a8-60f4-411f-9756-7eb360a05f02 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:11.083793 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a8e1b-efe0-4a8e-a4b0-97a5f794ca2c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:11.113356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:11.127486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d48bc-69c2-4625-8e3e-2a4da74923ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:11.128395 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d48bc-69c2-4625-8e3e-2a4da74923ed None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:11.128672 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d48bc-69c2-4625-8e3e-2a4da74923ed None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:11.129001 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d48bc-69c2-4625-8e3e-2a4da74923ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:11.143072 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d48bc-69c2-4625-8e3e-2a4da74923ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:11.149941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-316d48bc-69c2-4625-8e3e-2a4da74923ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:11.149941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:11.149941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:11.149941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:11.149941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:11.149941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:11.149941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:11.149941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:11.149941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:11.149941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:11.149941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:11.149941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:11.149941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:11.149941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:11.149941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:11.149941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:11.150853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:11.150853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:11.150853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:11.150853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:11.150853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:11.150853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:11.150853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:11.150853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:11.150853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:11.150853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:11.150853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:11.150853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:11.150853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:11.150853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:11.150853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:11.150853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:11.151671 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:11.151671 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:11.151671 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:11.151671 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:11.151671 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:11.151671 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:11.151671 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:11.151671 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:11.151671 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:11.151671 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:11.151671 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:11.151671 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:11.151671 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 124/248] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:11.153144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db09b18-ea84-4ed8-82fd-2eb1ed33bca9 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:11.153851 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db09b18-ea84-4ed8-82fd-2eb1ed33bca9 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:11.153851 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db09b18-ea84-4ed8-82fd-2eb1ed33bca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:11.176465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-9db09b18-ea84-4ed8-82fd-2eb1ed33bca9 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:11.177673 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9db09b18-ea84-4ed8-82fd-2eb1ed33bca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:11.207370 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4c8a8-60f4-411f-9756-7eb360a05f02 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:11.253372 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 125/249] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:11.254233 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db09b18-ea84-4ed8-82fd-2eb1ed33bca9 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:11.258667 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9551df5-5e4a-471a-9317-b3e395e6e8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:11.259611 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9551df5-5e4a-471a-9317-b3e395e6e8ea None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:11.260125 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9551df5-5e4a-471a-9317-b3e395e6e8ea None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:11.260591 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9551df5-5e4a-471a-9317-b3e395e6e8ea None admin] PATH_INFO: `/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:11.262980 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9551df5-5e4a-471a-9317-b3e395e6e8ea None admin] RBAC: Authorizing `identity:get_project(project_id=99c39c8f77a74048b8069e2cf0f83ae6)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:11.266060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9551df5-5e4a-471a-9317-b3e395e6e8ea None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:11.268955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 125/250] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:11 2022] GET /identity/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:11.275601 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9551df5-5e4a-471a-9317-b3e395e6e8ea None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:11.396331 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:11.461018 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db09b18-ea84-4ed8-82fd-2eb1ed33bca9 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:11.488285 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9551df5-5e4a-471a-9317-b3e395e6e8ea None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:11.497710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:11.523086 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcedbc3-e406-4c0f-af1b-58c079c71331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:11.524300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcedbc3-e406-4c0f-af1b-58c079c71331 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:11.525200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcedbc3-e406-4c0f-af1b-58c079c71331 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:11.525626 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcedbc3-e406-4c0f-af1b-58c079c71331 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:11.526525 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcedbc3-e406-4c0f-af1b-58c079c71331 None admin] RBAC: Adding query filter params (name=member) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:11.526991 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcedbc3-e406-4c0f-af1b-58c079c71331 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:11.528708 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcedbc3-e406-4c0f-af1b-58c079c71331 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:11.545746 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 126/251] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:51:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:11.547081 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c27d1-865d-4bcb-8c0a-863bec73e6ae None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:11.547372 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c27d1-865d-4bcb-8c0a-863bec73e6ae None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:11.547675 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c27d1-865d-4bcb-8c0a-863bec73e6ae None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:11.549518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 127/252] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:11.554246 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c27d1-865d-4bcb-8c0a-863bec73e6ae None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:11.589544 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638df5c-19c8-4093-8e3a-1ba766767863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:11.590161 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638df5c-19c8-4093-8e3a-1ba766767863 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:11.590445 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638df5c-19c8-4093-8e3a-1ba766767863 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:11.590713 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638df5c-19c8-4093-8e3a-1ba766767863 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:11.591442 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638df5c-19c8-4093-8e3a-1ba766767863 None admin] RBAC: Adding query filter params (role.id=f330ff772cca47c4b89a406784e00800, scope.project.id=99c39c8f77a74048b8069e2cf0f83ae6, user.id=630e0cb2a8a044b48e65e21a6aa81418) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:11.592118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638df5c-19c8-4093-8e3a-1ba766767863 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:11.593815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638df5c-19c8-4093-8e3a-1ba766767863 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:11.601820 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 126/253] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:51:11 2022] GET /identity/v3/role_assignments?user.id=630e0cb2a8a044b48e65e21a6aa81418&scope.project.id=99c39c8f77a74048b8069e2cf0f83ae6&role.id=f330ff772cca47c4b89a406784e00800 => generated 259 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:11.603117 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d64463-79ca-45cd-a5ae-62ad18da51a1 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:11.603403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d64463-79ca-45cd-a5ae-62ad18da51a1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:11.603718 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d64463-79ca-45cd-a5ae-62ad18da51a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:11.628537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f4d64463-79ca-45cd-a5ae-62ad18da51a1 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:11.630280 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f4d64463-79ca-45cd-a5ae-62ad18da51a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:11.701387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c27d1-865d-4bcb-8c0a-863bec73e6ae None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:11.759123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 127/254] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:11.760058 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d64463-79ca-45cd-a5ae-62ad18da51a1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:11.769437 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:11.788370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68c342-420e-4aec-b20a-9424ed872859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:11.789025 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68c342-420e-4aec-b20a-9424ed872859 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:11.789302 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68c342-420e-4aec-b20a-9424ed872859 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:11.789573 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68c342-420e-4aec-b20a-9424ed872859 None admin] PATH_INFO: `/v3/users/484f60d655cd421c927b28176787975d` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:11.790810 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe68c342-420e-4aec-b20a-9424ed872859 None admin] RBAC: Authorizing `identity:get_user(user_id=484f60d655cd421c927b28176787975d)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:11.792317 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe68c342-420e-4aec-b20a-9424ed872859 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:11.794471 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 128/255] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:51:11 2022] GET /identity/v3/users/484f60d655cd421c927b28176787975d => generated 297 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:11.800030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68c342-420e-4aec-b20a-9424ed872859 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:11.990604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:12.005531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d64463-79ca-45cd-a5ae-62ad18da51a1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:12.039214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68c342-420e-4aec-b20a-9424ed872859 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:12.086511 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4bfbed-4706-4388-a22d-611e56cd1b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:12.087168 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bfbed-4706-4388-a22d-611e56cd1b8a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:12.087452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bfbed-4706-4388-a22d-611e56cd1b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:12.087772 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bfbed-4706-4388-a22d-611e56cd1b8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:12.109137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb045430-076e-4a03-9cf7-bfb9c0f30eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:12.110137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb045430-076e-4a03-9cf7-bfb9c0f30eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:12.110609 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb045430-076e-4a03-9cf7-bfb9c0f30eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:12.111072 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb045430-076e-4a03-9cf7-bfb9c0f30eb8 None admin] PATH_INFO: `/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:12.116248 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb045430-076e-4a03-9cf7-bfb9c0f30eb8 None admin] RBAC: Authorizing `identity:get_project(project_id=99c39c8f77a74048b8069e2cf0f83ae6)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:12.116481 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4bfbed-4706-4388-a22d-611e56cd1b8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:12.118133 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb045430-076e-4a03-9cf7-bfb9c0f30eb8 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:12.120055 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 129/256] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:11 2022] GET /identity/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6 => generated 302 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:12.121363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-ed4bfbed-4706-4388-a22d-611e56cd1b8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:51:12.121363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:12.121363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:12.121363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:12.121363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:12.121363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:12.121363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:12.121363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:12.121363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:12.121363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:12.121363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:12.121363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:12.121363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:12.121363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:12.121363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:12.121363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:12.122230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:12.122230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:12.122230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:12.122230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:12.122230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:12.122230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:12.122230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:12.122230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:12.122230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:12.122230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:12.122230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:12.122230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:12.122230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:12.122230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:12.122230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:12.122230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:12.123026 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:12.123026 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:12.123026 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:12.123026 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:12.123026 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:12.123026 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:12.123026 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:12.123026 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:12.123026 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:12.123026 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:12.123026 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:51:12.123026 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:12.123026 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 128/257] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:51:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:12.128448 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb045430-076e-4a03-9cf7-bfb9c0f30eb8 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:12.128731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4bfbed-4706-4388-a22d-611e56cd1b8a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:12.247977 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:12.316159 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb045430-076e-4a03-9cf7-bfb9c0f30eb8 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:12.343221 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4bfbed-4706-4388-a22d-611e56cd1b8a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:12.390842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69853ccf-230e-4222-accc-1b7ff39c5eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:12.391531 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69853ccf-230e-4222-accc-1b7ff39c5eb2 None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:12.391833 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69853ccf-230e-4222-accc-1b7ff39c5eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:12.392108 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69853ccf-230e-4222-accc-1b7ff39c5eb2 None admin] PATH_INFO: `/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6/users/484f60d655cd421c927b28176787975d/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:12.397479 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69853ccf-230e-4222-accc-1b7ff39c5eb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=99c39c8f77a74048b8069e2cf0f83ae6, user_id=484f60d655cd421c927b28176787975d, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:12.399815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69853ccf-230e-4222-accc-1b7ff39c5eb2 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:12.407045 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d12c27-b8dc-4a10-88ed-538af0c01adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:12.407756 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d12c27-b8dc-4a10-88ed-538af0c01adf None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:12.408183 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d12c27-b8dc-4a10-88ed-538af0c01adf None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:12.408472 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d12c27-b8dc-4a10-88ed-538af0c01adf None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:12.409140 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d12c27-b8dc-4a10-88ed-538af0c01adf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:12.409438 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d12c27-b8dc-4a10-88ed-538af0c01adf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:12.410832 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d12c27-b8dc-4a10-88ed-538af0c01adf None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:12.425181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 130/258] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 00:51:12 2022] PUT /identity/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6/users/484f60d655cd421c927b28176787975d/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:51:12.426035 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69853ccf-230e-4222-accc-1b7ff39c5eb2 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:12.428974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 129/259] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:12.435266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d12c27-b8dc-4a10-88ed-538af0c01adf None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:12.488056 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:12.679913 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69853ccf-230e-4222-accc-1b7ff39c5eb2 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:12.706351 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d12c27-b8dc-4a10-88ed-538af0c01adf None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:12.722043 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:12.739658 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188e3c5-187b-4c7d-99b8-974ca49e8dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:12.740630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188e3c5-187b-4c7d-99b8-974ca49e8dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:12.741020 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188e3c5-187b-4c7d-99b8-974ca49e8dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:12.741401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188e3c5-187b-4c7d-99b8-974ca49e8dd4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:12.748030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f0adfc-c45a-48e7-b618-4e390b783f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:12.749210 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0adfc-c45a-48e7-b618-4e390b783f7d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:12.749634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0adfc-c45a-48e7-b618-4e390b783f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:12.750049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0adfc-c45a-48e7-b618-4e390b783f7d None admin] PATH_INFO: `/v3/users/1c485f5f331144bf8d9c5ed6e804cc29` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:12.751544 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f0adfc-c45a-48e7-b618-4e390b783f7d None admin] RBAC: Authorizing `identity:get_user(user_id=1c485f5f331144bf8d9c5ed6e804cc29)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:12.753349 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f0adfc-c45a-48e7-b618-4e390b783f7d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:12.759725 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 130/260] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:51:12 2022] GET /identity/v3/users/1c485f5f331144bf8d9c5ed6e804cc29 => generated 283 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:12.761483 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f0adfc-c45a-48e7-b618-4e390b783f7d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:12.764742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188e3c5-187b-4c7d-99b8-974ca49e8dd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:12.770959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-7188e3c5-187b-4c7d-99b8-974ca49e8dd4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:51:12.770959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:12.770959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:12.770959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:12.770959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:12.770959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:12.770959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:12.770959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:12.770959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:12.770959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:12.770959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:12.770959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:12.770959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:12.770959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:12.770959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:12.770959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:12.771872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:12.771872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:12.771872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:12.771872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:12.771872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:12.771872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:12.771872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:12.771872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:12.771872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:12.771872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:12.771872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:12.771872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:12.771872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:12.771872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:12.771872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:12.771872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:12.774464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:12.774464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:12.774464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:12.774464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:12.774464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:12.774464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:12.774464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:12.774464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:12.774464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:12.774464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:12.774464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:51:12.774464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:12.774464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 131/261] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 00:51:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:12.778282 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188e3c5-187b-4c7d-99b8-974ca49e8dd4 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:12.887898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:12.932357 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f0adfc-c45a-48e7-b618-4e390b783f7d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:12.986630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188e3c5-187b-4c7d-99b8-974ca49e8dd4 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:13.009301 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e5046-6ca8-4a62-93da-e01315867be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:13.009932 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e5046-6ca8-4a62-93da-e01315867be6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:13.010213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e5046-6ca8-4a62-93da-e01315867be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:13.010487 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e5046-6ca8-4a62-93da-e01315867be6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:13.087402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bbc5ac-1b03-4430-b5ca-16a0913a143f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:13.088139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbc5ac-1b03-4430-b5ca-16a0913a143f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:13.088422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbc5ac-1b03-4430-b5ca-16a0913a143f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:13.088699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbc5ac-1b03-4430-b5ca-16a0913a143f None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:13.089377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bbc5ac-1b03-4430-b5ca-16a0913a143f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:13.089682 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bbc5ac-1b03-4430-b5ca-16a0913a143f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:13.091074 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bbc5ac-1b03-4430-b5ca-16a0913a143f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:13.109936 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 132/262] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:51:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:13.115735 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bbc5ac-1b03-4430-b5ca-16a0913a143f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:13.124144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e5046-6ca8-4a62-93da-e01315867be6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:13.125650 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e5046-6ca8-4a62-93da-e01315867be6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:13.223988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:13.251747 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:13.275200 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 131/263] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 14 00:51:13.286384 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e5046-6ca8-4a62-93da-e01315867be6 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:13.304987 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bbc5ac-1b03-4430-b5ca-16a0913a143f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:13.341551 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e09dba1-b8f9-4a5b-890c-c44021c342fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:13.342546 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09dba1-b8f9-4a5b-890c-c44021c342fa None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:13.343022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09dba1-b8f9-4a5b-890c-c44021c342fa None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:13.343491 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09dba1-b8f9-4a5b-890c-c44021c342fa None admin] PATH_INFO: `/v3/users/1c485f5f331144bf8d9c5ed6e804cc29` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:13.344926 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e09dba1-b8f9-4a5b-890c-c44021c342fa None admin] RBAC: Authorizing `identity:get_user(user_id=1c485f5f331144bf8d9c5ed6e804cc29)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:13.347383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e09dba1-b8f9-4a5b-890c-c44021c342fa None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:13.349586 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 133/264] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:51:13 2022] GET /identity/v3/users/1c485f5f331144bf8d9c5ed6e804cc29 => generated 283 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:13.360450 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e09dba1-b8f9-4a5b-890c-c44021c342fa None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:13.450511 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e5046-6ca8-4a62-93da-e01315867be6 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:13.535037 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a2705-6d5e-4ea8-b9a3-754ec9dfe172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:13.535770 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a2705-6d5e-4ea8-b9a3-754ec9dfe172 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:13.536060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a2705-6d5e-4ea8-b9a3-754ec9dfe172 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:13.536341 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a2705-6d5e-4ea8-b9a3-754ec9dfe172 None admin] PATH_INFO: `/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:13.537598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a2705-6d5e-4ea8-b9a3-754ec9dfe172 None admin] RBAC: Authorizing `identity:get_project(project_id=99c39c8f77a74048b8069e2cf0f83ae6)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:13.539136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a2705-6d5e-4ea8-b9a3-754ec9dfe172 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:13.541028 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 132/265] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:13 2022] GET /identity/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6 => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:13.548677 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a2705-6d5e-4ea8-b9a3-754ec9dfe172 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:13.562112 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e09dba1-b8f9-4a5b-890c-c44021c342fa None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:13.593393 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:13.631404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b78fd-680d-400b-9fb0-c32730049a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:13.632167 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b78fd-680d-400b-9fb0-c32730049a81 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:13.632452 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b78fd-680d-400b-9fb0-c32730049a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:13.632728 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b78fd-680d-400b-9fb0-c32730049a81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:13.664469 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:13.746886 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26b78fd-680d-400b-9fb0-c32730049a81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:13.748352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26b78fd-680d-400b-9fb0-c32730049a81 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:13.819376 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a2705-6d5e-4ea8-b9a3-754ec9dfe172 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:13.861734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 134/266] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:51:13.863435 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729c60f-6bab-43dd-ab26-76eedc8044db None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:13.864191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729c60f-6bab-43dd-ab26-76eedc8044db None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:13.864686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729c60f-6bab-43dd-ab26-76eedc8044db None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:13.866408 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 135/267] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:13.871624 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80099d7-7bb5-44a8-ba84-74cbb8068c57 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:13.872246 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80099d7-7bb5-44a8-ba84-74cbb8068c57 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:13.872728 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80099d7-7bb5-44a8-ba84-74cbb8068c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:13.893280 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-c80099d7-7bb5-44a8-ba84-74cbb8068c57 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:13.894874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c80099d7-7bb5-44a8-ba84-74cbb8068c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:13.911976 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:13.924525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9bec5a-1daf-4620-a9ed-7f68fa0ca2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:13.925164 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bec5a-1daf-4620-a9ed-7f68fa0ca2a2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:13.925457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bec5a-1daf-4620-a9ed-7f68fa0ca2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:13.925740 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bec5a-1daf-4620-a9ed-7f68fa0ca2a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:13.926485 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9bec5a-1daf-4620-a9ed-7f68fa0ca2a2 None admin] RBAC: Adding query filter params (role.id=ada268ea63354a0297070369dfe3c6cc, scope.project.id=99c39c8f77a74048b8069e2cf0f83ae6, user.id=1c485f5f331144bf8d9c5ed6e804cc29) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:13.926804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9bec5a-1daf-4620-a9ed-7f68fa0ca2a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:13.928304 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9bec5a-1daf-4620-a9ed-7f68fa0ca2a2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:13.945983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 133/268] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:51:13 2022] GET /identity/v3/role_assignments?user.id=1c485f5f331144bf8d9c5ed6e804cc29&scope.project.id=99c39c8f77a74048b8069e2cf0f83ae6&role.id=ada268ea63354a0297070369dfe3c6cc => generated 608 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:13.947054 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9bec5a-1daf-4620-a9ed-7f68fa0ca2a2 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:13.973562 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 136/269] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:51:14.117547 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09388e-e634-4050-ac07-3a9d313e1a41 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:14.118168 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09388e-e634-4050-ac07-3a9d313e1a41 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:14.118669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09388e-e634-4050-ac07-3a9d313e1a41 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:14.120242 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 137/270] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:51:14.134478 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9bec5a-1daf-4620-a9ed-7f68fa0ca2a2 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:14.141446 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc2ef7-dc14-4db5-a4d3-f83f54aea1f8 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:14.142087 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc2ef7-dc14-4db5-a4d3-f83f54aea1f8 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:14.142655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc2ef7-dc14-4db5-a4d3-f83f54aea1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:14.166868 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:14.185683 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-7edc2ef7-dc14-4db5-a4d3-f83f54aea1f8 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:14.186421 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd25b3c-a2ca-4431-922d-ed86764cee60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:14.186895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7edc2ef7-dc14-4db5-a4d3-f83f54aea1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:14.187103 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd25b3c-a2ca-4431-922d-ed86764cee60 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:14.187337 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd25b3c-a2ca-4431-922d-ed86764cee60 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:14.187644 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd25b3c-a2ca-4431-922d-ed86764cee60 None admin] PATH_INFO: `/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:14.188997 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd25b3c-a2ca-4431-922d-ed86764cee60 None admin] RBAC: Authorizing `identity:get_project(project_id=99c39c8f77a74048b8069e2cf0f83ae6)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:14.190803 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd25b3c-a2ca-4431-922d-ed86764cee60 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:14.197377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 134/271] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:13 2022] GET /identity/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:14.198862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd25b3c-a2ca-4431-922d-ed86764cee60 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:14.305343 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd25b3c-a2ca-4431-922d-ed86764cee60 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:14.350964 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5a4bd-bc8e-4b82-9558-2da167f97354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:14.353280 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5a4bd-bc8e-4b82-9558-2da167f97354 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:14.353732 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5a4bd-bc8e-4b82-9558-2da167f97354 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:14.354146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5a4bd-bc8e-4b82-9558-2da167f97354 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:14.356041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5a4bd-bc8e-4b82-9558-2da167f97354 None admin] RBAC: Adding query filter params (role.id=89a3bc954b0a45bfa40a16db0ad2e51c, scope.project.id=99c39c8f77a74048b8069e2cf0f83ae6, user.id=1c485f5f331144bf8d9c5ed6e804cc29) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:14.357239 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5a4bd-bc8e-4b82-9558-2da167f97354 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:14.358848 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5a4bd-bc8e-4b82-9558-2da167f97354 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:14.373207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 135/272] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:51:14 2022] GET /identity/v3/role_assignments?user.id=1c485f5f331144bf8d9c5ed6e804cc29&scope.project.id=99c39c8f77a74048b8069e2cf0f83ae6&role.id=89a3bc954b0a45bfa40a16db0ad2e51c => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:14.381743 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 138/273] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:51:14.422911 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5a4bd-bc8e-4b82-9558-2da167f97354 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:14.487350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:14.526606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f47e1c-1764-4bf6-8190-ea4e0994e8f1 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:14.526978 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f47e1c-1764-4bf6-8190-ea4e0994e8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:14.527178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f47e1c-1764-4bf6-8190-ea4e0994e8f1 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:14.528852 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 139/274] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:51:14.538145 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8644d9c-dee6-47c0-a7b0-f88c50838e94 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:14.538431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8644d9c-dee6-47c0-a7b0-f88c50838e94 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:14.538710 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8644d9c-dee6-47c0-a7b0-f88c50838e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:14.555485 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5a4bd-bc8e-4b82-9558-2da167f97354 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:14.569671 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e8644d9c-dee6-47c0-a7b0-f88c50838e94 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:14.570874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e8644d9c-dee6-47c0-a7b0-f88c50838e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:14.597635 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961df6e-a326-4b03-a5ce-aa23da08ae45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:14.598582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961df6e-a326-4b03-a5ce-aa23da08ae45 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:14.598862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961df6e-a326-4b03-a5ce-aa23da08ae45 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:14.599138 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961df6e-a326-4b03-a5ce-aa23da08ae45 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:14.612598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3961df6e-a326-4b03-a5ce-aa23da08ae45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:14.618381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-3961df6e-a326-4b03-a5ce-aa23da08ae45 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:14.618381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:14.618381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:14.618381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:14.618381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:14.618381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:14.618381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:14.618381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:14.618381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:14.618381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:14.618381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:14.618381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:14.618381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:14.618381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:14.618381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:14.618381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:14.619326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:14.619326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:14.619326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:14.619326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:14.619326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:14.619326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:14.619326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:14.619326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:14.619326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:14.619326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:14.619326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:14.619326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:14.619326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:14.619326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:14.619326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:14.619326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:14.620536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:14.620536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:14.620536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:14.620536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:14.620536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:14.620536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:14.620536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:14.620536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:14.620536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:14.620536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:14.620536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:14.620536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:14.620536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 136/275] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:14.631040 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961df6e-a326-4b03-a5ce-aa23da08ae45 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:14.701484 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 140/276] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:51:14.773873 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961df6e-a326-4b03-a5ce-aa23da08ae45 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:14.782846 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:14.800248 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8644d9c-dee6-47c0-a7b0-f88c50838e94 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:14.809590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:14.813484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79b94b-f58d-427b-b660-3194f3de1dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:14.814121 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79b94b-f58d-427b-b660-3194f3de1dec None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:14.814445 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79b94b-f58d-427b-b660-3194f3de1dec None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:14.814726 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79b94b-f58d-427b-b660-3194f3de1dec None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:14.815834 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a79b94b-f58d-427b-b660-3194f3de1dec None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:14.816337 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a79b94b-f58d-427b-b660-3194f3de1dec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:14.818774 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a79b94b-f58d-427b-b660-3194f3de1dec None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:14.827863 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 137/277] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:51:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:14.836661 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79b94b-f58d-427b-b660-3194f3de1dec None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:14.937215 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8644d9c-dee6-47c0-a7b0-f88c50838e94 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:14.965056 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79b94b-f58d-427b-b660-3194f3de1dec None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:14.990960 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d190b9d2-183c-47a9-a749-a180d7375d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:14.991622 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190b9d2-183c-47a9-a749-a180d7375d93 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:14.991905 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190b9d2-183c-47a9-a749-a180d7375d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:14.992180 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190b9d2-183c-47a9-a749-a180d7375d93 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:14.995833 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d190b9d2-183c-47a9-a749-a180d7375d93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:14.997306 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d190b9d2-183c-47a9-a749-a180d7375d93 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:14.998904 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 141/278] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:15.004724 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d190b9d2-183c-47a9-a749-a180d7375d93 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:15.014528 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf8e31-2318-4d7a-9685-d65d159baefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:15.015747 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf8e31-2318-4d7a-9685-d65d159baefd None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:15.016226 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf8e31-2318-4d7a-9685-d65d159baefd None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:15.016692 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf8e31-2318-4d7a-9685-d65d159baefd None admin] PATH_INFO: `/v3/users/853950b179914339b9daebe3fe0df616` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:15.031700 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf8e31-2318-4d7a-9685-d65d159baefd None admin] RBAC: Authorizing `identity:get_user(user_id=853950b179914339b9daebe3fe0df616)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:15.033469 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf8e31-2318-4d7a-9685-d65d159baefd None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:15.035767 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 138/279] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:51:14 2022] GET /identity/v3/users/853950b179914339b9daebe3fe0df616 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:15.040535 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf8e31-2318-4d7a-9685-d65d159baefd None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:15.069918 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:15.086803 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:15.113117 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d190b9d2-183c-47a9-a749-a180d7375d93 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:15.148593 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf8e31-2318-4d7a-9685-d65d159baefd None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:15.148779 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88242029-3f52-40e8-aae2-98d2eedfc2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:15.149257 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88242029-3f52-40e8-aae2-98d2eedfc2aa None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:15.149534 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88242029-3f52-40e8-aae2-98d2eedfc2aa None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:15.149804 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88242029-3f52-40e8-aae2-98d2eedfc2aa None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:15.150624 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88242029-3f52-40e8-aae2-98d2eedfc2aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:15.152886 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88242029-3f52-40e8-aae2-98d2eedfc2aa None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:15.177186 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 142/280] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 00:51:15 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:51:15.178400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88242029-3f52-40e8-aae2-98d2eedfc2aa None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:15.203510 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d34cf9-7def-4b9a-bfad-db82ae14b6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:15.205362 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d34cf9-7def-4b9a-bfad-db82ae14b6df None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:15.205872 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d34cf9-7def-4b9a-bfad-db82ae14b6df None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:15.206711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d34cf9-7def-4b9a-bfad-db82ae14b6df None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:15.225081 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d34cf9-7def-4b9a-bfad-db82ae14b6df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:15.232525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-53d34cf9-7def-4b9a-bfad-db82ae14b6df None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 00:51:15.232525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:15.232525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:15.232525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:15.232525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:15.232525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:15.232525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:15.232525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:15.232525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:15.232525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:15.232525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:15.232525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:15.232525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:15.232525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:15.232525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:15.232525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:15.233582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:15.233582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:15.233582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:15.233582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:15.233582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:15.233582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:15.233582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:15.233582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:15.233582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:15.233582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:15.233582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:15.233582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:15.233582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:15.233582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:15.233582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:15.233582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:15.234378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:15.234378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:15.234378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:15.234378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:15.234378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:15.234378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:15.234378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:15.234378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:15.234378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:15.234378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:15.234378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 00:51:15.234378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:15.237756 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 139/281] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:15.244019 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d34cf9-7def-4b9a-bfad-db82ae14b6df None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:15.320938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88242029-3f52-40e8-aae2-98d2eedfc2aa None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:15.358798 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:15.367162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:15.368439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb41e56-a50b-477e-a90d-52c2ee59a99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:15.369435 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb41e56-a50b-477e-a90d-52c2ee59a99a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:15.369944 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb41e56-a50b-477e-a90d-52c2ee59a99a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:15.370418 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb41e56-a50b-477e-a90d-52c2ee59a99a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:15.371489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb41e56-a50b-477e-a90d-52c2ee59a99a None admin] RBAC: Adding query filter params (role.id=5d9045e45840408a92ceca71e9184ba2, user.id=853950b179914339b9daebe3fe0df616, scope.system=all) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:15.372034 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb41e56-a50b-477e-a90d-52c2ee59a99a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:15.374484 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb41e56-a50b-477e-a90d-52c2ee59a99a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:15.382009 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 143/282] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Sep 14 00:51:15 2022] GET /identity/v3/role_assignments?user.id=853950b179914339b9daebe3fe0df616&scope.system=all&role.id=5d9045e45840408a92ceca71e9184ba2 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:15.391221 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d34cf9-7def-4b9a-bfad-db82ae14b6df None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:15.434165 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698357b-2d7c-4523-b938-91cdb23c81da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:15.434787 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698357b-2d7c-4523-b938-91cdb23c81da None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:15.435065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698357b-2d7c-4523-b938-91cdb23c81da None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:15.435337 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698357b-2d7c-4523-b938-91cdb23c81da None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:15.436051 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698357b-2d7c-4523-b938-91cdb23c81da None admin] RBAC: Adding query filter params (name=reader) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:15.436424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698357b-2d7c-4523-b938-91cdb23c81da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:15.437939 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698357b-2d7c-4523-b938-91cdb23c81da None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:15.449712 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 140/283] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:51:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:15.457092 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb41e56-a50b-477e-a90d-52c2ee59a99a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:15.584837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177dda9c-d6fb-450f-9cc4-6e7e8a9c451f None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:15.585045 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177dda9c-d6fb-450f-9cc4-6e7e8a9c451f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:15.585285 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177dda9c-d6fb-450f-9cc4-6e7e8a9c451f None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:15.586732 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 141/284] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:51:15.592097 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34f66f-6657-47eb-83e6-257829106826 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:15.592445 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34f66f-6657-47eb-83e6-257829106826 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:15.592721 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34f66f-6657-47eb-83e6-257829106826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:15.599063 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb41e56-a50b-477e-a90d-52c2ee59a99a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:15.624716 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ef34f66f-6657-47eb-83e6-257829106826 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:15.626069 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ef34f66f-6657-47eb-83e6-257829106826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:15.643146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef4e09-bf48-45c4-a1f2-88e32c35b838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:15.645737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef4e09-bf48-45c4-a1f2-88e32c35b838 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:15.646022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef4e09-bf48-45c4-a1f2-88e32c35b838 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:15.646299 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef4e09-bf48-45c4-a1f2-88e32c35b838 None admin] PATH_INFO: `/v3/users/630e0cb2a8a044b48e65e21a6aa81418` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:15.647590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef4e09-bf48-45c4-a1f2-88e32c35b838 None admin] RBAC: Authorizing `identity:get_user(user_id=630e0cb2a8a044b48e65e21a6aa81418)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:15.650121 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef4e09-bf48-45c4-a1f2-88e32c35b838 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:15.652698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 144/285] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:51:15 2022] GET /identity/v3/users/630e0cb2a8a044b48e65e21a6aa81418 => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:15.661209 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef4e09-bf48-45c4-a1f2-88e32c35b838 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:15.718740 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 142/286] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:51:15.739539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:15.783476 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef4e09-bf48-45c4-a1f2-88e32c35b838 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:15.819108 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ed2e2-17ac-42ed-a6cd-7722d4fd498f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:15.819888 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ed2e2-17ac-42ed-a6cd-7722d4fd498f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:15.820050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ed2e2-17ac-42ed-a6cd-7722d4fd498f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:15.820330 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ed2e2-17ac-42ed-a6cd-7722d4fd498f None admin] PATH_INFO: `/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:15.821473 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ed2e2-17ac-42ed-a6cd-7722d4fd498f None admin] RBAC: Authorizing `identity:get_project(project_id=99c39c8f77a74048b8069e2cf0f83ae6)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:15.822931 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ed2e2-17ac-42ed-a6cd-7722d4fd498f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:15.825015 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 145/287] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:15 2022] GET /identity/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:15.831411 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34f66f-6657-47eb-83e6-257829106826 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:15.944433 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:15.972915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34f66f-6657-47eb-83e6-257829106826 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:16.025736 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99f62d-d2ef-4e9b-8963-d5888ed94dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:16.026551 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99f62d-d2ef-4e9b-8963-d5888ed94dd3 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:16.026837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99f62d-d2ef-4e9b-8963-d5888ed94dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:16.027587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99f62d-d2ef-4e9b-8963-d5888ed94dd3 None admin] PATH_INFO: `/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6/users/630e0cb2a8a044b48e65e21a6aa81418/roles/f330ff772cca47c4b89a406784e00800` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:16.029638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf99f62d-d2ef-4e9b-8963-d5888ed94dd3 None admin] RBAC: Authorizing `identity:create_grant(project_id=99c39c8f77a74048b8069e2cf0f83ae6, user_id=630e0cb2a8a044b48e65e21a6aa81418, role_id=f330ff772cca47c4b89a406784e00800)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:16.031110 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf99f62d-d2ef-4e9b-8963-d5888ed94dd3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:16.054534 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 143/288] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 00:51:15 2022] PUT /identity/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6/users/630e0cb2a8a044b48e65e21a6aa81418/roles/f330ff772cca47c4b89a406784e00800 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:51:16.059386 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ed2e2-17ac-42ed-a6cd-7722d4fd498f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:16.234076 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ed2e2-17ac-42ed-a6cd-7722d4fd498f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:16.280911 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:16.288522 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65930f-557e-480b-b0ab-2188a2e9fde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:16.289180 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65930f-557e-480b-b0ab-2188a2e9fde5 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:16.289462 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65930f-557e-480b-b0ab-2188a2e9fde5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:16.289743 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65930f-557e-480b-b0ab-2188a2e9fde5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:16.308058 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e65930f-557e-480b-b0ab-2188a2e9fde5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:16.313699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-4e65930f-557e-480b-b0ab-2188a2e9fde5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:16.313699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:16.313699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:16.313699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:16.313699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:16.313699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:16.313699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:16.313699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:16.313699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:16.313699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:16.313699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:16.313699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:16.313699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:16.313699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:16.313699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:16.313699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:16.314869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:16.314869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:16.314869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:16.314869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:16.314869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:16.314869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:16.314869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:16.314869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:16.314869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:16.314869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:16.314869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:16.314869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:16.314869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:16.314869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:16.314869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:16.314869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:16.315908 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:16.315908 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:16.315908 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:16.315908 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:16.315908 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:16.315908 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:16.315908 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:16.315908 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:16.315908 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:16.315908 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:16.315908 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:16.315908 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:16.315908 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 146/289] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:16.321596 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99f62d-d2ef-4e9b-8963-d5888ed94dd3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:16.448224 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99f62d-d2ef-4e9b-8963-d5888ed94dd3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:16.486006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271575b1-baae-4a12-bb67-a8251c792814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:16.487114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271575b1-baae-4a12-bb67-a8251c792814 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:16.487399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271575b1-baae-4a12-bb67-a8251c792814 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:16.487712 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271575b1-baae-4a12-bb67-a8251c792814 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:16.488393 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271575b1-baae-4a12-bb67-a8251c792814 None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:16.488704 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271575b1-baae-4a12-bb67-a8251c792814 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:16.490107 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271575b1-baae-4a12-bb67-a8251c792814 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:16.499656 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 144/290] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:51:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:16.505637 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65930f-557e-480b-b0ab-2188a2e9fde5 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:16.679460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65930f-557e-480b-b0ab-2188a2e9fde5 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:16.732868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12b47f-0128-4ced-9320-fc949e14c346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:16.733525 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12b47f-0128-4ced-9320-fc949e14c346 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:16.733812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12b47f-0128-4ced-9320-fc949e14c346 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:16.734090 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12b47f-0128-4ced-9320-fc949e14c346 None admin] PATH_INFO: `/v3/users/484f60d655cd421c927b28176787975d` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:16.736947 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12b47f-0128-4ced-9320-fc949e14c346 None admin] RBAC: Authorizing `identity:get_user(user_id=484f60d655cd421c927b28176787975d)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:16.738368 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12b47f-0128-4ced-9320-fc949e14c346 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:16.740323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 147/291] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:51:16 2022] GET /identity/v3/users/484f60d655cd421c927b28176787975d => generated 297 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:16.746283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271575b1-baae-4a12-bb67-a8251c792814 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:16.749589 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:16.916484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271575b1-baae-4a12-bb67-a8251c792814 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:16.968354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd564bf-30d1-430b-8f8c-e5b645feb5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:16.968969 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd564bf-30d1-430b-8f8c-e5b645feb5c4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:16.969366 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd564bf-30d1-430b-8f8c-e5b645feb5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:16.969854 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd564bf-30d1-430b-8f8c-e5b645feb5c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:17.062104 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd564bf-30d1-430b-8f8c-e5b645feb5c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:17.063514 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd564bf-30d1-430b-8f8c-e5b645feb5c4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:17.067860 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:17.189627 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 145/292] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:51:17.195025 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12b47f-0128-4ced-9320-fc949e14c346 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:17.293495 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:17.344294 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12b47f-0128-4ced-9320-fc949e14c346 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:17.383746 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b6910-7907-4d78-9575-88ddf847d148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:17.384388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b6910-7907-4d78-9575-88ddf847d148 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:17.384666 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b6910-7907-4d78-9575-88ddf847d148 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:17.384939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b6910-7907-4d78-9575-88ddf847d148 None admin] PATH_INFO: `/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:17.392289 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4b6910-7907-4d78-9575-88ddf847d148 None admin] RBAC: Authorizing `identity:get_project(project_id=99c39c8f77a74048b8069e2cf0f83ae6)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:17.394666 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4b6910-7907-4d78-9575-88ddf847d148 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:17.396726 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 148/293] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:17 2022] GET /identity/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:17.402948 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd564bf-30d1-430b-8f8c-e5b645feb5c4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:17.543596 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd564bf-30d1-430b-8f8c-e5b645feb5c4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:17.545431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:17.582974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e53d68-8019-4465-97a3-dbc256fed257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:17.583601 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e53d68-8019-4465-97a3-dbc256fed257 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:17.583884 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e53d68-8019-4465-97a3-dbc256fed257 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:17.584157 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e53d68-8019-4465-97a3-dbc256fed257 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:17.584876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e53d68-8019-4465-97a3-dbc256fed257 None admin] RBAC: Adding query filter params (role.id=5d9045e45840408a92ceca71e9184ba2, scope.project.id=99c39c8f77a74048b8069e2cf0f83ae6, user.id=484f60d655cd421c927b28176787975d) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:17.585235 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e53d68-8019-4465-97a3-dbc256fed257 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:17.586731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e53d68-8019-4465-97a3-dbc256fed257 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:17.593042 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 146/294] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:51:17 2022] GET /identity/v3/role_assignments?user.id=484f60d655cd421c927b28176787975d&scope.project.id=99c39c8f77a74048b8069e2cf0f83ae6&role.id=5d9045e45840408a92ceca71e9184ba2 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:17.949270 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb20780-fc8a-45d4-b140-45d0a0f6b5ee None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:17.949556 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb20780-fc8a-45d4-b140-45d0a0f6b5ee None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:17.949835 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb20780-fc8a-45d4-b140-45d0a0f6b5ee None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:17.951171 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 149/295] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:17.957228 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c09e8e-0838-458e-bb19-5df15c5b260f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:17.957711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c09e8e-0838-458e-bb19-5df15c5b260f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:17.957891 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c09e8e-0838-458e-bb19-5df15c5b260f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:17.976977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-40c09e8e-0838-458e-bb19-5df15c5b260f None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:17.978161 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-40c09e8e-0838-458e-bb19-5df15c5b260f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:18.054835 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 147/296] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:18.327944 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274eb82-629b-4307-8cef-f03d9af2911f None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:18.328429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274eb82-629b-4307-8cef-f03d9af2911f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:18.328429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274eb82-629b-4307-8cef-f03d9af2911f None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:18.329952 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 150/297] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:18.336235 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e40ef3-4fbb-4963-b8a2-3209de54c4a4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:18.336855 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e40ef3-4fbb-4963-b8a2-3209de54c4a4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:18.337366 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e40ef3-4fbb-4963-b8a2-3209de54c4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:18.357279 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-76e40ef3-4fbb-4963-b8a2-3209de54c4a4 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:18.358804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-76e40ef3-4fbb-4963-b8a2-3209de54c4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:18.360006 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274eb82-629b-4307-8cef-f03d9af2911f None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:18.431667 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 148/298] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:18.452665 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ff537-4a95-48bc-be60-17d757bacadf None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:18.452963 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ff537-4a95-48bc-be60-17d757bacadf None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:18.453224 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ff537-4a95-48bc-be60-17d757bacadf None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:18.454744 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 149/299] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:51:18.460147 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4873ce-1f5c-45b1-83a3-02e3f68cfda6 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:18.460432 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4873ce-1f5c-45b1-83a3-02e3f68cfda6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:18.460710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4873ce-1f5c-45b1-83a3-02e3f68cfda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:18.481583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0d4873ce-1f5c-45b1-83a3-02e3f68cfda6 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:18.482772 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0d4873ce-1f5c-45b1-83a3-02e3f68cfda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:18.483901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274eb82-629b-4307-8cef-f03d9af2911f None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:18.509361 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:18.520622 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:18.532079 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402560e3-731c-468d-a0b4-05640218c600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:18.532817 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402560e3-731c-468d-a0b4-05640218c600 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:18.533676 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402560e3-731c-468d-a0b4-05640218c600 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:18.534013 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402560e3-731c-468d-a0b4-05640218c600 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:18.547649 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402560e3-731c-468d-a0b4-05640218c600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:18.554047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-402560e3-731c-468d-a0b4-05640218c600 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 00:51:18.554047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:18.554047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:18.554047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:18.554047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:18.554047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:18.554047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:18.554047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:18.554047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:18.554047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:18.554047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:18.554047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:18.554047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:18.554047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:18.554047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:18.554047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:18.554974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:18.554974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:18.554974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:18.554974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:18.554974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:18.554974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:18.554974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:18.554974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:18.554974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:18.554974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:18.554974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:18.554974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:18.554974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:18.554974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:18.554974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:18.554974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:18.555820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:18.555820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:18.555820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:18.555820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:18.555820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:18.555820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:18.555820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:18.555820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:18.555820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:18.555820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:18.555820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 00:51:18.555820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:18.555820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 151/300] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:51:18.560870 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402560e3-731c-468d-a0b4-05640218c600 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:18.570769 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 150/301] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:18.672731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402560e3-731c-468d-a0b4-05640218c600 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:18.709413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f3567-cf41-4b4b-be7d-83eacf9f00c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:18.710380 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f3567-cf41-4b4b-be7d-83eacf9f00c3 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:18.710881 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f3567-cf41-4b4b-be7d-83eacf9f00c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:18.711361 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f3567-cf41-4b4b-be7d-83eacf9f00c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:18.712570 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f3567-cf41-4b4b-be7d-83eacf9f00c3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:18.713098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f3567-cf41-4b4b-be7d-83eacf9f00c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:18.714683 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f3567-cf41-4b4b-be7d-83eacf9f00c3 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:18.718958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4873ce-1f5c-45b1-83a3-02e3f68cfda6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:18.725759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 152/302] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:51:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:18.731266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f3567-cf41-4b4b-be7d-83eacf9f00c3 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:18.797663 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:18.858080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4873ce-1f5c-45b1-83a3-02e3f68cfda6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:18.865013 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f3567-cf41-4b4b-be7d-83eacf9f00c3 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:18.897724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d9ec4-217f-4503-9ca7-07800f8c98a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:18.898527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d9ec4-217f-4503-9ca7-07800f8c98a4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:18.898878 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d9ec4-217f-4503-9ca7-07800f8c98a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:18.899219 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d9ec4-217f-4503-9ca7-07800f8c98a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:18.900465 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d9ec4-217f-4503-9ca7-07800f8c98a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:18.901863 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d9ec4-217f-4503-9ca7-07800f8c98a4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:18.901964 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13250e-3f38-4db7-bb86-3985beb367ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:18.902518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13250e-3f38-4db7-bb86-3985beb367ab None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:18.902806 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13250e-3f38-4db7-bb86-3985beb367ab None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:18.903080 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13250e-3f38-4db7-bb86-3985beb367ab None admin] PATH_INFO: `/v3/users/e163c58a399d4ac4a2d2816c086a325c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:18.903627 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 151/303] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:18.904228 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d9ec4-217f-4503-9ca7-07800f8c98a4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:18.922947 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13250e-3f38-4db7-bb86-3985beb367ab None admin] RBAC: Authorizing `identity:get_user(user_id=e163c58a399d4ac4a2d2816c086a325c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:18.925073 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13250e-3f38-4db7-bb86-3985beb367ab None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:18.926750 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 153/304] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:51:18 2022] GET /identity/v3/users/e163c58a399d4ac4a2d2816c086a325c => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:18.927504 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13250e-3f38-4db7-bb86-3985beb367ab None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:18.940582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:19.019577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d9ec4-217f-4503-9ca7-07800f8c98a4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:19.030956 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13250e-3f38-4db7-bb86-3985beb367ab None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:19.038970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:19.053880 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd4cea1-3d92-4a54-ae28-649a7bd97721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:19.054600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4cea1-3d92-4a54-ae28-649a7bd97721 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:19.054947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4cea1-3d92-4a54-ae28-649a7bd97721 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:19.055286 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4cea1-3d92-4a54-ae28-649a7bd97721 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:19.065364 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd4cea1-3d92-4a54-ae28-649a7bd97721 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:19.066799 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc1f5aa-f3b3-4fe6-8c56-6bc5b2595836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:19.067411 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1f5aa-f3b3-4fe6-8c56-6bc5b2595836 None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:19.067733 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1f5aa-f3b3-4fe6-8c56-6bc5b2595836 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:19.068013 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1f5aa-f3b3-4fe6-8c56-6bc5b2595836 None admin] PATH_INFO: `/v3/groups` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:19.068657 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc1f5aa-f3b3-4fe6-8c56-6bc5b2595836 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:19.070087 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc1f5aa-f3b3-4fe6-8c56-6bc5b2595836 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:19.070185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-bfd4cea1-3d92-4a54-ae28-649a7bd97721 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:19.070185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:19.070185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:19.070185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:19.070185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:19.070185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:19.070185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:19.070185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:19.070185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:19.070185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:19.070185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:19.070185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:19.070185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:19.070185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:19.070185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:19.070185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:19.071010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:19.071010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:19.071010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:19.071010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:19.071010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:19.071010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:19.071010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:19.071010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:19.071010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:19.071010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:19.071010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:19.071010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:19.071010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:19.071010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:19.071010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:19.071010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:19.071850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:19.071850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:19.071850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:19.071850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:19.071850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:19.071850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:19.071850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:19.071850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:19.071850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:19.071850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:19.071850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:19.071850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:19.071850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 152/305] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:19.076581 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd4cea1-3d92-4a54-ae28-649a7bd97721 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:19.086077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 154/306] 10.222.0.42 () {60 vars in 1134 bytes} [Wed Sep 14 00:51:18 2022] POST /identity/v3/groups => generated 229 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:51:19.087317 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8a01c-01c2-469e-9304-1fd87aaac58f None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:19.087635 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8a01c-01c2-469e-9304-1fd87aaac58f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:19.087912 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8a01c-01c2-469e-9304-1fd87aaac58f None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:19.089380 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 155/307] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:19.090008 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f8a01c-01c2-469e-9304-1fd87aaac58f None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:19.179387 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:19.200299 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd4cea1-3d92-4a54-ae28-649a7bd97721 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:19.205701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f8a01c-01c2-469e-9304-1fd87aaac58f None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:19.240963 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e1f99-8499-4d4c-b7f9-83f3a0d78f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:19.241619 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e1f99-8499-4d4c-b7f9-83f3a0d78f78 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:19.241897 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e1f99-8499-4d4c-b7f9-83f3a0d78f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:19.242170 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e1f99-8499-4d4c-b7f9-83f3a0d78f78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:19.242515 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095cba9-df9b-4ca3-8c0f-ca142b3ca99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:19.242911 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e1f99-8499-4d4c-b7f9-83f3a0d78f78 None admin] RBAC: Adding query filter params (role.id=f330ff772cca47c4b89a406784e00800, user.id=e163c58a399d4ac4a2d2816c086a325c, scope.system=all) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:19.243126 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095cba9-df9b-4ca3-8c0f-ca142b3ca99f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:19.243224 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e1f99-8499-4d4c-b7f9-83f3a0d78f78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:19.243407 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095cba9-df9b-4ca3-8c0f-ca142b3ca99f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:19.243697 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095cba9-df9b-4ca3-8c0f-ca142b3ca99f None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:19.244367 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9095cba9-df9b-4ca3-8c0f-ca142b3ca99f None admin] RBAC: Adding query filter params (name=member) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:19.244667 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9095cba9-df9b-4ca3-8c0f-ca142b3ca99f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:19.244760 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e1f99-8499-4d4c-b7f9-83f3a0d78f78 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:19.246040 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9095cba9-df9b-4ca3-8c0f-ca142b3ca99f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:19.259576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 153/308] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Sep 14 00:51:19 2022] GET /identity/v3/role_assignments?user.id=e163c58a399d4ac4a2d2816c086a325c&scope.system=all&role.id=f330ff772cca47c4b89a406784e00800 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:19.259576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87293836-eab6-45fa-be4b-744201b30ff4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:19.259576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87293836-eab6-45fa-be4b-744201b30ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:19.259576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87293836-eab6-45fa-be4b-744201b30ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:19.266295 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 156/309] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:51:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:19.272465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095cba9-df9b-4ca3-8c0f-ca142b3ca99f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:19.276266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-87293836-eab6-45fa-be4b-744201b30ff4 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:19.277509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-87293836-eab6-45fa-be4b-744201b30ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:19.310780 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:19.367403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 154/310] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:19.413023 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095cba9-df9b-4ca3-8c0f-ca142b3ca99f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:19.452647 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2de369-673f-4dcf-ad0c-6499b8a0a6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:19.453216 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2de369-673f-4dcf-ad0c-6499b8a0a6ab None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:19.453498 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2de369-673f-4dcf-ad0c-6499b8a0a6ab None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:19.453775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2de369-673f-4dcf-ad0c-6499b8a0a6ab None admin] PATH_INFO: `/v3/users/853950b179914339b9daebe3fe0df616` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:19.456000 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2de369-673f-4dcf-ad0c-6499b8a0a6ab None admin] RBAC: Authorizing `identity:get_user(user_id=853950b179914339b9daebe3fe0df616)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:19.457244 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2de369-673f-4dcf-ad0c-6499b8a0a6ab None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:19.458833 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 157/311] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:51:19 2022] GET /identity/v3/users/853950b179914339b9daebe3fe0df616 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:19.464610 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87293836-eab6-45fa-be4b-744201b30ff4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:19.547231 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:19.570490 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87293836-eab6-45fa-be4b-744201b30ff4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:19.604619 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e265a037-c4a2-4dea-a3b6-74c00060ba7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:19.605280 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265a037-c4a2-4dea-a3b6-74c00060ba7d None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:19.605559 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265a037-c4a2-4dea-a3b6-74c00060ba7d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:19.605837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265a037-c4a2-4dea-a3b6-74c00060ba7d None admin] PATH_INFO: `/v3/system/users/853950b179914339b9daebe3fe0df616/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:19.607376 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e265a037-c4a2-4dea-a3b6-74c00060ba7d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=853950b179914339b9daebe3fe0df616, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:19.608792 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e265a037-c4a2-4dea-a3b6-74c00060ba7d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:19.619019 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 155/312] 10.222.0.42 () {58 vars in 1334 bytes} [Wed Sep 14 00:51:19 2022] PUT /identity/v3/system/users/853950b179914339b9daebe3fe0df616/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 00:51:19.674295 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2de369-673f-4dcf-ad0c-6499b8a0a6ab None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:19.784395 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:19.790999 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2de369-673f-4dcf-ad0c-6499b8a0a6ab None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:19.827336 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370150f-fd13-4d85-bbb9-23de3d0b1cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:19.828013 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370150f-fd13-4d85-bbb9-23de3d0b1cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:19.828292 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370150f-fd13-4d85-bbb9-23de3d0b1cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:19.828567 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370150f-fd13-4d85-bbb9-23de3d0b1cd4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:19.839312 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a370150f-fd13-4d85-bbb9-23de3d0b1cd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:19.844191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-a370150f-fd13-4d85-bbb9-23de3d0b1cd4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 00:51:19.844191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:19.844191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:19.844191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:19.844191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:19.844191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:19.844191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:19.844191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:19.844191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:19.844191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:19.844191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:19.844191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:19.844191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:19.844191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:19.844191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:19.844191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:19.845065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:19.845065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:19.845065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:19.845065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:19.845065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:19.845065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:19.845065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:19.845065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:19.845065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:19.845065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:19.845065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:19.845065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:19.845065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:19.845065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:19.845065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:19.845065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:19.846737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:19.846737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:19.846737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:19.846737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:19.846737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:19.846737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:19.846737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:19.846737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:19.846737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:19.846737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:19.846737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 00:51:19.846737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:19.846737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 158/313] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:19.852109 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e265a037-c4a2-4dea-a3b6-74c00060ba7d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:19.957305 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e265a037-c4a2-4dea-a3b6-74c00060ba7d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:19.992171 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae9abb-fde1-46c1-85a3-14a2336e9fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:19.992789 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae9abb-fde1-46c1-85a3-14a2336e9fef None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:19.993073 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae9abb-fde1-46c1-85a3-14a2336e9fef None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:19.993353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae9abb-fde1-46c1-85a3-14a2336e9fef None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:19.994024 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae9abb-fde1-46c1-85a3-14a2336e9fef None admin] RBAC: Adding query filter params (name=reader) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:19.994331 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae9abb-fde1-46c1-85a3-14a2336e9fef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:19.995734 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae9abb-fde1-46c1-85a3-14a2336e9fef None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:20.005191 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 156/314] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:51:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:20.010936 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370150f-fd13-4d85-bbb9-23de3d0b1cd4 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:20.118319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370150f-fd13-4d85-bbb9-23de3d0b1cd4 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:20.156740 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fa2fa-190b-4147-9c20-718fb2e00b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:20.157382 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fa2fa-190b-4147-9c20-718fb2e00b96 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:20.157665 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fa2fa-190b-4147-9c20-718fb2e00b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:20.157999 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fa2fa-190b-4147-9c20-718fb2e00b96 None admin] PATH_INFO: `/v3/users/630e0cb2a8a044b48e65e21a6aa81418` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:20.159908 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1fa2fa-190b-4147-9c20-718fb2e00b96 None admin] RBAC: Authorizing `identity:get_user(user_id=630e0cb2a8a044b48e65e21a6aa81418)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:20.161365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1fa2fa-190b-4147-9c20-718fb2e00b96 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:20.163344 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 159/315] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:51:20 2022] GET /identity/v3/users/630e0cb2a8a044b48e65e21a6aa81418 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:20.168888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae9abb-fde1-46c1-85a3-14a2336e9fef None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:20.173152 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:20.273299 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae9abb-fde1-46c1-85a3-14a2336e9fef None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:20.309827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4bf0c-0770-4e4b-9e1c-81fea20bb798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:20.310444 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4bf0c-0770-4e4b-9e1c-81fea20bb798 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:20.310727 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4bf0c-0770-4e4b-9e1c-81fea20bb798 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:20.311005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4bf0c-0770-4e4b-9e1c-81fea20bb798 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:20.381989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4bf0c-0770-4e4b-9e1c-81fea20bb798 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:20.383535 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4bf0c-0770-4e4b-9e1c-81fea20bb798 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:20.386337 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:20.492328 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 157/316] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:51:20.497725 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fa2fa-190b-4147-9c20-718fb2e00b96 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:20.526544 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:20.617923 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fa2fa-190b-4147-9c20-718fb2e00b96 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:20.653945 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f15dfff-6ea8-4501-9f80-13e41b36f0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:20.654504 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15dfff-6ea8-4501-9f80-13e41b36f0d4 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:20.654788 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15dfff-6ea8-4501-9f80-13e41b36f0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:20.655067 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15dfff-6ea8-4501-9f80-13e41b36f0d4 None admin] PATH_INFO: `/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:20.656226 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f15dfff-6ea8-4501-9f80-13e41b36f0d4 None admin] RBAC: Authorizing `identity:get_project(project_id=99c39c8f77a74048b8069e2cf0f83ae6)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:20.658493 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f15dfff-6ea8-4501-9f80-13e41b36f0d4 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:20.660398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 160/317] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:20 2022] GET /identity/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:20.666208 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4bf0c-0770-4e4b-9e1c-81fea20bb798 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:20.769529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4bf0c-0770-4e4b-9e1c-81fea20bb798 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:20.771987 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:20.806045 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c1e0b-4903-4bb2-95cc-36e8d9c96bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:20.806660 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c1e0b-4903-4bb2-95cc-36e8d9c96bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:20.806937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c1e0b-4903-4bb2-95cc-36e8d9c96bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:20.807209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c1e0b-4903-4bb2-95cc-36e8d9c96bb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:20.807969 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c1e0b-4903-4bb2-95cc-36e8d9c96bb0 None admin] RBAC: Adding query filter params (role.id=f330ff772cca47c4b89a406784e00800, scope.project.id=99c39c8f77a74048b8069e2cf0f83ae6, user.id=630e0cb2a8a044b48e65e21a6aa81418) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:20.808276 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c1e0b-4903-4bb2-95cc-36e8d9c96bb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:20.809718 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c1e0b-4903-4bb2-95cc-36e8d9c96bb0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:20.816113 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 158/318] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:51:20 2022] GET /identity/v3/role_assignments?user.id=630e0cb2a8a044b48e65e21a6aa81418&scope.project.id=99c39c8f77a74048b8069e2cf0f83ae6&role.id=f330ff772cca47c4b89a406784e00800 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:21.451138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bf3e0-bab9-45d7-b8fa-389a64ded200 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:21.451540 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bf3e0-bab9-45d7-b8fa-389a64ded200 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:21.451920 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bf3e0-bab9-45d7-b8fa-389a64ded200 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:21.453505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 161/319] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:21.459301 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d04a40-fec7-4276-9548-293c0d795d94 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:21.459816 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d04a40-fec7-4276-9548-293c0d795d94 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:21.459891 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d04a40-fec7-4276-9548-293c0d795d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:21.483108 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-51d04a40-fec7-4276-9548-293c0d795d94 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:21.484362 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-51d04a40-fec7-4276-9548-293c0d795d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:21.559592 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 159/320] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:21.647727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aa7e8-69ee-490f-8774-9ee178da3620 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:21.648231 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aa7e8-69ee-490f-8774-9ee178da3620 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:21.648432 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aa7e8-69ee-490f-8774-9ee178da3620 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:21.649819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 162/321] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:21.654823 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15046e89-7c67-467b-b2ee-1a2e42d16e1f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:21.655118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15046e89-7c67-467b-b2ee-1a2e42d16e1f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:21.655397 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15046e89-7c67-467b-b2ee-1a2e42d16e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:21.674260 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-15046e89-7c67-467b-b2ee-1a2e42d16e1f None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:21.675451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-15046e89-7c67-467b-b2ee-1a2e42d16e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:21.741832 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 160/322] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:21.831342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034aa7e8-69ee-490f-8774-9ee178da3620 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:21.897152 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac5317-afb3-40f5-8b18-5633a318273a None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:21.897438 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac5317-afb3-40f5-8b18-5633a318273a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:21.897718 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac5317-afb3-40f5-8b18-5633a318273a None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:21.899063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 161/323] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:21.904210 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baafab0-37cb-4d6e-b0a6-f2f7322c987e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:21.904499 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baafab0-37cb-4d6e-b0a6-f2f7322c987e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:21.904777 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baafab0-37cb-4d6e-b0a6-f2f7322c987e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:21.923373 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2baafab0-37cb-4d6e-b0a6-f2f7322c987e None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:21.924620 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2baafab0-37cb-4d6e-b0a6-f2f7322c987e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:21.927732 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:21.940151 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034aa7e8-69ee-490f-8774-9ee178da3620 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:21.950694 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:21.975728 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a3a65-cdf2-4b78-ab41-a52133eb4a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:21.976368 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a3a65-cdf2-4b78-ab41-a52133eb4a28 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:21.976652 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a3a65-cdf2-4b78-ab41-a52133eb4a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:21.976919 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a3a65-cdf2-4b78-ab41-a52133eb4a28 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:21.978319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a3a65-cdf2-4b78-ab41-a52133eb4a28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:21.979803 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a3a65-cdf2-4b78-ab41-a52133eb4a28 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:21.983608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 163/324] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:21.986924 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a3a65-cdf2-4b78-ab41-a52133eb4a28 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:22.001128 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 162/325] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:51:22.019954 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baafab0-37cb-4d6e-b0a6-f2f7322c987e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:22.098156 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a3a65-cdf2-4b78-ab41-a52133eb4a28 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:22.131512 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baafab0-37cb-4d6e-b0a6-f2f7322c987e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:22.135439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e0284-1725-4e4e-b384-1125c0dce1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:22.136104 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e0284-1725-4e4e-b384-1125c0dce1b8 None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:22.136388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e0284-1725-4e4e-b384-1125c0dce1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:22.136661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e0284-1725-4e4e-b384-1125c0dce1b8 None admin] PATH_INFO: `/v3/groups` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:22.137302 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e0284-1725-4e4e-b384-1125c0dce1b8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:22.139596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e0284-1725-4e4e-b384-1125c0dce1b8 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:22.156655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 164/326] 10.222.0.42 () {60 vars in 1134 bytes} [Wed Sep 14 00:51:21 2022] POST /identity/v3/groups => generated 226 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 00:51:22.169812 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dceb85-4d3d-4a1e-ac7a-f08fffe3a5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:22.170503 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dceb85-4d3d-4a1e-ac7a-f08fffe3a5c4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:22.170780 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dceb85-4d3d-4a1e-ac7a-f08fffe3a5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:22.171052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dceb85-4d3d-4a1e-ac7a-f08fffe3a5c4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:22.184415 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dceb85-4d3d-4a1e-ac7a-f08fffe3a5c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:22.189425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-d6dceb85-4d3d-4a1e-ac7a-f08fffe3a5c4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 00:51:22.189425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:22.189425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:22.189425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:22.189425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:22.189425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:22.189425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:22.189425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:22.189425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:22.189425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:22.189425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:22.189425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:22.189425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:22.189425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:22.189425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:22.189425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:22.190386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:22.190386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:22.190386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:22.190386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:22.190386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:22.190386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:22.190386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:22.190386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:22.190386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:22.190386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:22.190386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:22.190386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:22.190386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:22.190386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:22.190386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:22.190386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:22.191213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:22.191213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:22.191213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:22.191213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:22.191213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:22.191213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:22.191213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:22.191213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:22.191213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:22.191213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:22.191213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 00:51:22.191213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:22.192487 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 163/327] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:51:22.197071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e0284-1725-4e4e-b384-1125c0dce1b8 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:22.222908 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:22.272671 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dceb85-4d3d-4a1e-ac7a-f08fffe3a5c4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:22.295273 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:22.313188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e0284-1725-4e4e-b384-1125c0dce1b8 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:22.354912 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10284d-8f98-4b49-a9a7-4108d49bb971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:22.355539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10284d-8f98-4b49-a9a7-4108d49bb971 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:22.355889 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10284d-8f98-4b49-a9a7-4108d49bb971 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:22.356173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10284d-8f98-4b49-a9a7-4108d49bb971 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:22.356849 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd10284d-8f98-4b49-a9a7-4108d49bb971 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:22.357154 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd10284d-8f98-4b49-a9a7-4108d49bb971 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:22.358543 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd10284d-8f98-4b49-a9a7-4108d49bb971 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:22.368239 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 165/328] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:51:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:22.373623 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10284d-8f98-4b49-a9a7-4108d49bb971 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:22.396779 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dceb85-4d3d-4a1e-ac7a-f08fffe3a5c4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:22.438770 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793b37e-ff00-464f-85bb-5acb44fd697f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:22.440092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793b37e-ff00-464f-85bb-5acb44fd697f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:22.440833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793b37e-ff00-464f-85bb-5acb44fd697f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:22.440943 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793b37e-ff00-464f-85bb-5acb44fd697f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:22.454631 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c793b37e-ff00-464f-85bb-5acb44fd697f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:22.461076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-c793b37e-ff00-464f-85bb-5acb44fd697f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:22.461076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:22.461076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:22.461076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:22.461076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:22.461076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:22.461076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:22.461076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:22.461076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:22.461076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:22.461076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:22.461076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:22.461076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:22.461076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:22.461076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:22.461076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:22.462406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:22.462406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:22.462406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:22.462406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:22.462406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:22.462406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:22.462406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:22.462406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:22.462406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:22.462406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:22.462406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:22.462406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:22.462406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:22.462406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:22.462406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:22.462406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:22.463217 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:22.463217 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:22.463217 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:22.463217 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:22.463217 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:22.463217 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:22.463217 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:22.463217 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:22.463217 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:22.463217 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:22.463217 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:22.463217 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:22.463217 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 164/329] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:51:22.470765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:22.471887 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793b37e-ff00-464f-85bb-5acb44fd697f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:22.496228 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10284d-8f98-4b49-a9a7-4108d49bb971 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:22.536241 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1eb039-d34a-47d9-99f9-8cb45585f577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:22.537659 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1eb039-d34a-47d9-99f9-8cb45585f577 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:22.538408 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1eb039-d34a-47d9-99f9-8cb45585f577 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:22.538879 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1eb039-d34a-47d9-99f9-8cb45585f577 None admin] PATH_INFO: `/v3/users/e163c58a399d4ac4a2d2816c086a325c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:22.540773 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1eb039-d34a-47d9-99f9-8cb45585f577 None admin] RBAC: Authorizing `identity:get_user(user_id=e163c58a399d4ac4a2d2816c086a325c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:22.542847 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1eb039-d34a-47d9-99f9-8cb45585f577 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:22.545098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 166/330] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:51:22 2022] GET /identity/v3/users/e163c58a399d4ac4a2d2816c086a325c => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:22.551797 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1eb039-d34a-47d9-99f9-8cb45585f577 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:22.567464 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:22.595089 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793b37e-ff00-464f-85bb-5acb44fd697f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:22.652648 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a4232-861a-4225-a040-961108c0aeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:22.653282 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a4232-861a-4225-a040-961108c0aeea None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:22.653552 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a4232-861a-4225-a040-961108c0aeea None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:22.653827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a4232-861a-4225-a040-961108c0aeea None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:22.654508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a4232-861a-4225-a040-961108c0aeea None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:22.654813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a4232-861a-4225-a040-961108c0aeea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:22.656254 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a4232-861a-4225-a040-961108c0aeea None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:22.669380 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 165/331] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:51:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:22.676554 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a4232-861a-4225-a040-961108c0aeea None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:22.695966 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1eb039-d34a-47d9-99f9-8cb45585f577 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:22.743616 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7ba4e-ff29-46ae-b2bb-f55a92df0fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:22.744630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7ba4e-ff29-46ae-b2bb-f55a92df0fa5 None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:22.745051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7ba4e-ff29-46ae-b2bb-f55a92df0fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:22.745464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7ba4e-ff29-46ae-b2bb-f55a92df0fa5 None admin] PATH_INFO: `/v3/system/users/e163c58a399d4ac4a2d2816c086a325c/roles/f330ff772cca47c4b89a406784e00800` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:22.748132 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7ba4e-ff29-46ae-b2bb-f55a92df0fa5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e163c58a399d4ac4a2d2816c086a325c, role_id=f330ff772cca47c4b89a406784e00800)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:22.749760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7ba4e-ff29-46ae-b2bb-f55a92df0fa5 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:22.763706 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 167/332] 10.222.0.42 () {58 vars in 1334 bytes} [Wed Sep 14 00:51:22 2022] PUT /identity/v3/system/users/e163c58a399d4ac4a2d2816c086a325c/roles/f330ff772cca47c4b89a406784e00800 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:51:22.807549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a4232-861a-4225-a040-961108c0aeea None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:22.857132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8799d-ca60-4358-8282-22c9477e0534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:22.857801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8799d-ca60-4358-8282-22c9477e0534 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:22.858087 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8799d-ca60-4358-8282-22c9477e0534 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:22.858365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8799d-ca60-4358-8282-22c9477e0534 None admin] PATH_INFO: `/v3/users/853950b179914339b9daebe3fe0df616` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:22.859739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8799d-ca60-4358-8282-22c9477e0534 None admin] RBAC: Authorizing `identity:get_user(user_id=853950b179914339b9daebe3fe0df616)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:22.861496 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8799d-ca60-4358-8282-22c9477e0534 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:22.863484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 166/333] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:51:22 2022] GET /identity/v3/users/853950b179914339b9daebe3fe0df616 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:22.869341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7ba4e-ff29-46ae-b2bb-f55a92df0fa5 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:22.901667 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:23.010501 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7ba4e-ff29-46ae-b2bb-f55a92df0fa5 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:23.051932 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d80cd7-ba7a-43ec-85d5-abe135e3aec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:23.052554 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d80cd7-ba7a-43ec-85d5-abe135e3aec1 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:23.052830 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d80cd7-ba7a-43ec-85d5-abe135e3aec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:23.053101 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d80cd7-ba7a-43ec-85d5-abe135e3aec1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:23.053828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d80cd7-ba7a-43ec-85d5-abe135e3aec1 None admin] RBAC: Adding query filter params (role.id=5d9045e45840408a92ceca71e9184ba2, user.id=853950b179914339b9daebe3fe0df616, scope.system=all) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:23.054135 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d80cd7-ba7a-43ec-85d5-abe135e3aec1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:23.055692 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d80cd7-ba7a-43ec-85d5-abe135e3aec1 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:23.070158 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 168/334] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Sep 14 00:51:22 2022] GET /identity/v3/role_assignments?user.id=853950b179914339b9daebe3fe0df616&scope.system=all&role.id=5d9045e45840408a92ceca71e9184ba2 => generated 510 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:25.408998 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec598e-ca93-4ffc-8cf7-2a29fbc26af8 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:25.408998 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec598e-ca93-4ffc-8cf7-2a29fbc26af8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:25.408998 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbec598e-ca93-4ffc-8cf7-2a29fbc26af8 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:25.410181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 167/335] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:25.421775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eaf6eb-3cda-4d08-9066-25907aa8227f None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:25.422287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eaf6eb-3cda-4d08-9066-25907aa8227f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:25.422287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eaf6eb-3cda-4d08-9066-25907aa8227f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:25.453455 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-16eaf6eb-3cda-4d08-9066-25907aa8227f None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:25.454692 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-16eaf6eb-3cda-4d08-9066-25907aa8227f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:25.465439 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3ffb9-2c46-484c-84b9-5e624cb58441 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:25.466202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3ffb9-2c46-484c-84b9-5e624cb58441 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:25.466546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3ffb9-2c46-484c-84b9-5e624cb58441 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:25.468317 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 168/336] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:25.487375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40de836-f9fe-4b3e-bc5c-6f0ad5128bb0 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:25.487715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40de836-f9fe-4b3e-bc5c-6f0ad5128bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:25.488132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40de836-f9fe-4b3e-bc5c-6f0ad5128bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:25.519600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b40de836-f9fe-4b3e-bc5c-6f0ad5128bb0 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:25.521063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b40de836-f9fe-4b3e-bc5c-6f0ad5128bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:25.523197 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:25.573544 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 169/337] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:25.596083 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165887b-2d6c-4e01-b302-2116dbffd1de None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:25.596377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165887b-2d6c-4e01-b302-2116dbffd1de None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:25.596621 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165887b-2d6c-4e01-b302-2116dbffd1de None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:25.598198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 170/338] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:25.603404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d2a64-d826-43b4-ad6a-b3eccfae43a2 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:25.603776 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d2a64-d826-43b4-ad6a-b3eccfae43a2 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:25.604059 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d2a64-d826-43b4-ad6a-b3eccfae43a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:25.615323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:25.619577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 169/339] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:51:25.632831 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-b82d2a64-d826-43b4-ad6a-b3eccfae43a2 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:25.634038 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b82d2a64-d826-43b4-ad6a-b3eccfae43a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:25.747071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 171/340] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:51:26.125032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40de836-f9fe-4b3e-bc5c-6f0ad5128bb0 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:26.137269 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d2a64-d826-43b4-ad6a-b3eccfae43a2 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:26.272575 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d2a64-d826-43b4-ad6a-b3eccfae43a2 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:26.312739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40de836-f9fe-4b3e-bc5c-6f0ad5128bb0 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:26.324345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:26.330493 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee9fcb-1c81-463e-8a54-2d5a074ec080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:26.331139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee9fcb-1c81-463e-8a54-2d5a074ec080 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:26.331419 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee9fcb-1c81-463e-8a54-2d5a074ec080 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:26.331743 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee9fcb-1c81-463e-8a54-2d5a074ec080 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:26.344739 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee9fcb-1c81-463e-8a54-2d5a074ec080 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:26.349963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-edee9fcb-1c81-463e-8a54-2d5a074ec080 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:26.349963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:26.349963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:26.349963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:26.349963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:26.349963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:26.349963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:26.349963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:26.349963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:26.349963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:26.349963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:26.349963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:26.349963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:26.349963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:26.349963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:26.349963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:26.350844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:26.350844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:26.350844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:26.350844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:26.350844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:26.350844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:26.350844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:26.350844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:26.350844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:26.350844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:26.350844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:26.350844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:26.350844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:26.350844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:26.350844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:26.350844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:26.351661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:26.351661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:26.351661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:26.351661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:26.351661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:26.351661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:26.351661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:26.351661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:26.351661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:26.351661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:26.351661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:26.351661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:26.351661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 172/341] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:26.352789 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79678cf3-ec2f-4dd7-88f1-f9c0177ab827 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:26.353513 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79678cf3-ec2f-4dd7-88f1-f9c0177ab827 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:26.353844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79678cf3-ec2f-4dd7-88f1-f9c0177ab827 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:26.358954 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230fbcd-9624-4504-8eb6-21d2d5a96a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:26.359944 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 173/342] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:26.360192 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230fbcd-9624-4504-8eb6-21d2d5a96a01 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:26.360485 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230fbcd-9624-4504-8eb6-21d2d5a96a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:26.360583 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79678cf3-ec2f-4dd7-88f1-f9c0177ab827 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:26.360766 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230fbcd-9624-4504-8eb6-21d2d5a96a01 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:26.370619 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:26.376046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230fbcd-9624-4504-8eb6-21d2d5a96a01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:26.381374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-e230fbcd-9624-4504-8eb6-21d2d5a96a01 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:51:26.381374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:26.381374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:26.381374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:26.381374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:26.381374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:26.381374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:26.381374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:26.381374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:26.381374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:26.381374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:26.381374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:26.381374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:26.381374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:26.381374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:26.381374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:26.383470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:26.383470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:26.383470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:26.383470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:26.383470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:26.383470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:26.383470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:26.383470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:26.383470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:26.383470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:26.383470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:26.383470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:26.383470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:26.383470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:26.383470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:26.383470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:26.385394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:26.385394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:26.385394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:26.385394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:26.385394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:26.385394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:26.385394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:26.385394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:26.385394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:26.385394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:26.385394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:51:26.385394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:26.385394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 170/343] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 00:51:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:26.385394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230fbcd-9624-4504-8eb6-21d2d5a96a01 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:26.503034 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230fbcd-9624-4504-8eb6-21d2d5a96a01 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:26.514803 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79678cf3-ec2f-4dd7-88f1-f9c0177ab827 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:26.542215 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d8b20-7e57-471a-9d72-9c5a60965bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:26.543088 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d8b20-7e57-471a-9d72-9c5a60965bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:26.543609 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d8b20-7e57-471a-9d72-9c5a60965bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:26.544020 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d8b20-7e57-471a-9d72-9c5a60965bb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:26.544829 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2d8b20-7e57-471a-9d72-9c5a60965bb1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:26.545523 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2d8b20-7e57-471a-9d72-9c5a60965bb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:26.547685 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2d8b20-7e57-471a-9d72-9c5a60965bb1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:26.557271 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672ec0e-6b0a-4641-87db-5699f2d9ab49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:26.557723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 171/344] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:51:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:26.558067 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672ec0e-6b0a-4641-87db-5699f2d9ab49 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:26.558441 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672ec0e-6b0a-4641-87db-5699f2d9ab49 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:26.558740 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672ec0e-6b0a-4641-87db-5699f2d9ab49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:26.559275 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409a488-d664-4e6f-b396-df3e805a29cc None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:26.559579 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409a488-d664-4e6f-b396-df3e805a29cc None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:26.559868 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409a488-d664-4e6f-b396-df3e805a29cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:26.574782 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0672ec0e-6b0a-4641-87db-5699f2d9ab49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:26.579908 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0409a488-d664-4e6f-b396-df3e805a29cc None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:26.580516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-0672ec0e-6b0a-4641-87db-5699f2d9ab49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:26.580516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:26.580516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:26.580516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:26.580516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:26.580516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:26.580516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:26.580516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:26.580516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:26.580516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:26.580516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:26.580516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:26.580516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:26.580516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:26.580516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:26.580516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:26.581472 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:26.581472 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:26.581472 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:26.581472 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:26.581472 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:26.581472 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:26.581472 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:26.581472 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:26.581472 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:26.581472 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:26.581472 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:26.581472 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:26.581472 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:26.581472 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:26.581472 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:26.581472 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:26.582763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:26.582763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:26.582763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:26.582763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:26.582763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:26.582763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:26.582763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:26.582763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:26.582763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:26.582763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:26.582763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:26.582763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:26.583378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0409a488-d664-4e6f-b396-df3e805a29cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:26.583449 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 174/345] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:26.583449 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672ec0e-6b0a-4641-87db-5699f2d9ab49 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:26.654425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:26.657205 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:26.672897 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 172/346] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:26.675637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7ac72-8945-4374-9bc2-6b3a5dd38a0e None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:26.675637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7ac72-8945-4374-9bc2-6b3a5dd38a0e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:26.675637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7ac72-8945-4374-9bc2-6b3a5dd38a0e None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:26.679763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 173/347] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:26.679763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7ac72-8945-4374-9bc2-6b3a5dd38a0e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:26.725571 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672ec0e-6b0a-4641-87db-5699f2d9ab49 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:26.769659 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20503b0-3e20-494e-a392-2078688f5e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:26.770351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20503b0-3e20-494e-a392-2078688f5e4c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:26.770632 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20503b0-3e20-494e-a392-2078688f5e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:26.770906 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20503b0-3e20-494e-a392-2078688f5e4c None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:26.772427 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20503b0-3e20-494e-a392-2078688f5e4c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:26.772740 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20503b0-3e20-494e-a392-2078688f5e4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:26.774306 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20503b0-3e20-494e-a392-2078688f5e4c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:26.784502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 175/348] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:51:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:26.785163 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20503b0-3e20-494e-a392-2078688f5e4c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:26.800416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7ac72-8945-4374-9bc2-6b3a5dd38a0e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:26.840156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f90591d-85ee-41bb-808e-372c8902b26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:26.841261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90591d-85ee-41bb-808e-372c8902b26c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:26.841542 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90591d-85ee-41bb-808e-372c8902b26c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:26.841811 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f90591d-85ee-41bb-808e-372c8902b26c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:26.906548 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20503b0-3e20-494e-a392-2078688f5e4c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:26.912891 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:26.929286 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:26.933234 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f90591d-85ee-41bb-808e-372c8902b26c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:26.936320 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f90591d-85ee-41bb-808e-372c8902b26c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:26.967136 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49549478-7345-4131-91c5-9866450a67cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:26.967852 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49549478-7345-4131-91c5-9866450a67cd None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:26.968072 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49549478-7345-4131-91c5-9866450a67cd None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:26.968341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49549478-7345-4131-91c5-9866450a67cd None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:26.969092 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49549478-7345-4131-91c5-9866450a67cd None admin] RBAC: Adding query filter params (name=member) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:26.969399 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49549478-7345-4131-91c5-9866450a67cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:26.971381 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49549478-7345-4131-91c5-9866450a67cd None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:26.980506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 176/349] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:51:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:26.981927 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af8c2f-5c05-42ce-9942-a504f502abef None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:26.982212 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af8c2f-5c05-42ce-9942-a504f502abef None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:26.982487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af8c2f-5c05-42ce-9942-a504f502abef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:27.004090 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-40af8c2f-5c05-42ce-9942-a504f502abef None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:27.005663 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-40af8c2f-5c05-42ce-9942-a504f502abef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:27.064086 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 174/350] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:51:27.065442 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f90591d-85ee-41bb-808e-372c8902b26c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:27.093320 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 177/351] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:27.094080 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af8c2f-5c05-42ce-9942-a504f502abef None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:27.177936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:27.190740 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f90591d-85ee-41bb-808e-372c8902b26c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:27.214253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af8c2f-5c05-42ce-9942-a504f502abef None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:27.225484 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:27.241749 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a7071-9fa9-442a-8665-cce31d7ab5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:27.242375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a7071-9fa9-442a-8665-cce31d7ab5b2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:27.242659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a7071-9fa9-442a-8665-cce31d7ab5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:27.242934 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a7071-9fa9-442a-8665-cce31d7ab5b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:27.252743 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cbcee-7f32-4013-82db-e7464bb810c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:27.253589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cbcee-7f32-4013-82db-e7464bb810c3 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:27.253968 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cbcee-7f32-4013-82db-e7464bb810c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:27.254244 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cbcee-7f32-4013-82db-e7464bb810c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:27.331069 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a7071-9fa9-442a-8665-cce31d7ab5b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:27.332568 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a7071-9fa9-442a-8665-cce31d7ab5b2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:27.334501 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6cbcee-7f32-4013-82db-e7464bb810c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:27.336279 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6cbcee-7f32-4013-82db-e7464bb810c3 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:27.447025 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:27.462261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 175/352] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:51:27.462261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a7071-9fa9-442a-8665-cce31d7ab5b2 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:27.463798 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 178/353] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:51:27.464633 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cbcee-7f32-4013-82db-e7464bb810c3 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:27.475799 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:27.587171 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cbcee-7f32-4013-82db-e7464bb810c3 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:27.594823 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a7071-9fa9-442a-8665-cce31d7ab5b2 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:27.632726 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124429c-e943-40a2-87c0-99deddc5834f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:27.633359 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124429c-e943-40a2-87c0-99deddc5834f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:27.633635 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124429c-e943-40a2-87c0-99deddc5834f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:27.633909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124429c-e943-40a2-87c0-99deddc5834f None admin] PATH_INFO: `/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:27.636466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124429c-e943-40a2-87c0-99deddc5834f None admin] RBAC: Authorizing `identity:get_project(project_id=99c39c8f77a74048b8069e2cf0f83ae6)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:27.639178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124429c-e943-40a2-87c0-99deddc5834f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:27.640008 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd694973-2d70-40bc-9fe3-510b9316a901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:27.641122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd694973-2d70-40bc-9fe3-510b9316a901 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:27.641404 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd694973-2d70-40bc-9fe3-510b9316a901 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:27.641683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd694973-2d70-40bc-9fe3-510b9316a901 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:27.643050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 179/354] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:27 2022] GET /identity/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:27.645074 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b991a0-cc6e-4727-8898-70ebeae75b81 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:27.645355 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b991a0-cc6e-4727-8898-70ebeae75b81 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:27.645631 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b991a0-cc6e-4727-8898-70ebeae75b81 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:27.647222 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 180/355] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:27.648053 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b991a0-cc6e-4727-8898-70ebeae75b81 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:27.657577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd694973-2d70-40bc-9fe3-510b9316a901 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:27.662598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-dd694973-2d70-40bc-9fe3-510b9316a901 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 00:51:27.662598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:27.662598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:27.662598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:27.662598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:27.662598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:27.662598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:27.662598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:27.662598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:27.662598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:27.662598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:27.662598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:27.662598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:27.662598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:27.662598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:27.662598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:27.663763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:27.663763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:27.663763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:27.663763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:27.663763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:27.663763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:27.663763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:27.663763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:27.663763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:27.663763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:27.663763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:27.663763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:27.663763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:27.663763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:27.663763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:27.663763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:27.664585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:27.664585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:27.664585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:27.664585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:27.664585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:27.664585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:27.664585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:27.664585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:27.664585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:27.664585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:27.664585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 00:51:27.664585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:27.664585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 176/356] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:27.665243 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd694973-2d70-40bc-9fe3-510b9316a901 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:27.728549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:27.740918 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:27.773086 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b991a0-cc6e-4727-8898-70ebeae75b81 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:27.801090 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd694973-2d70-40bc-9fe3-510b9316a901 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:27.809407 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e555b6-15b7-4078-ab8d-9af078547301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:27.811333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e555b6-15b7-4078-ab8d-9af078547301 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:27.811333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e555b6-15b7-4078-ab8d-9af078547301 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:27.811333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e555b6-15b7-4078-ab8d-9af078547301 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:27.822387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e555b6-15b7-4078-ab8d-9af078547301 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:27.827232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-f8e555b6-15b7-4078-ab8d-9af078547301 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:51:27.827232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:27.827232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:27.827232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:27.827232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:27.827232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:27.827232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:27.827232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:27.827232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:27.827232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:27.827232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:27.827232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:27.827232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:27.827232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:27.827232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:27.827232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:27.828319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:27.828319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:27.828319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:27.828319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:27.828319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:27.828319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:27.828319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:27.828319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:27.828319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:27.828319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:27.828319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:27.828319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:27.828319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:27.828319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:27.828319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:27.828319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:27.829134 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:27.829134 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:27.829134 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:27.829134 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:27.829134 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:27.829134 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:27.829134 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:27.829134 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:27.829134 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:27.829134 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:27.829134 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:51:27.829134 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:27.829134 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 181/357] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 00:51:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:27.831453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e555b6-15b7-4078-ab8d-9af078547301 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:27.849367 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39445a32-02c1-4db1-ba39-51aabd62f840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:27.850315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39445a32-02c1-4db1-ba39-51aabd62f840 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:27.850767 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39445a32-02c1-4db1-ba39-51aabd62f840 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:27.851213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39445a32-02c1-4db1-ba39-51aabd62f840 None admin] PATH_INFO: `/v3/projects/ac5d363867c44c349f5be02f57cd5958` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:27.852600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39445a32-02c1-4db1-ba39-51aabd62f840 None admin] RBAC: Authorizing `identity:get_project(project_id=ac5d363867c44c349f5be02f57cd5958)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:27.854117 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39445a32-02c1-4db1-ba39-51aabd62f840 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:27.859865 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 177/358] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:27 2022] GET /identity/v3/projects/ac5d363867c44c349f5be02f57cd5958 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:27.861481 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39445a32-02c1-4db1-ba39-51aabd62f840 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:27.943763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e555b6-15b7-4078-ab8d-9af078547301 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:27.980789 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39445a32-02c1-4db1-ba39-51aabd62f840 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:27.987876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1431e63e-6c43-43bc-9d29-45f773d47756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:27.988741 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431e63e-6c43-43bc-9d29-45f773d47756 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:27.989601 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431e63e-6c43-43bc-9d29-45f773d47756 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:27.990031 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1431e63e-6c43-43bc-9d29-45f773d47756 None admin] PATH_INFO: `/v3/projects/ac5d363867c44c349f5be02f57cd5958` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:27.991542 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431e63e-6c43-43bc-9d29-45f773d47756 None admin] RBAC: Authorizing `identity:get_project(project_id=ac5d363867c44c349f5be02f57cd5958)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:27.993351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1431e63e-6c43-43bc-9d29-45f773d47756 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:27.995489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 182/359] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:27 2022] GET /identity/v3/projects/ac5d363867c44c349f5be02f57cd5958 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:27.996754 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b7195-9528-45dd-90cb-89b6a46d9292 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:27.997033 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b7195-9528-45dd-90cb-89b6a46d9292 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:27.997401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b7195-9528-45dd-90cb-89b6a46d9292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:28.000626 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:28.004809 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:28.020561 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-c37b7195-9528-45dd-90cb-89b6a46d9292 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:28.021858 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c37b7195-9528-45dd-90cb-89b6a46d9292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:28.024258 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584f39f-f9cb-4f3a-baa4-cc5985c7e442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:28.025742 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584f39f-f9cb-4f3a-baa4-cc5985c7e442 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:28.026026 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584f39f-f9cb-4f3a-baa4-cc5985c7e442 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:28.026302 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584f39f-f9cb-4f3a-baa4-cc5985c7e442 None admin] PATH_INFO: `/v3/groups/543cc929fec0446e956024ebce66e97b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:28.033451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c584f39f-f9cb-4f3a-baa4-cc5985c7e442 None admin] RBAC: Authorizing `identity:get_group(group_id=543cc929fec0446e956024ebce66e97b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:28.034914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c584f39f-f9cb-4f3a-baa4-cc5985c7e442 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:28.036862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 178/360] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 00:51:27 2022] GET /identity/v3/groups/543cc929fec0446e956024ebce66e97b => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:28.037385 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584f39f-f9cb-4f3a-baa4-cc5985c7e442 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:28.100903 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 183/361] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:28.101891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b7195-9528-45dd-90cb-89b6a46d9292 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:28.161023 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584f39f-f9cb-4f3a-baa4-cc5985c7e442 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:28.208233 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b76c79-8d35-419b-aa7d-ad627da991c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:28.208861 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b76c79-8d35-419b-aa7d-ad627da991c4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:28.209148 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b76c79-8d35-419b-aa7d-ad627da991c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:28.209421 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b76c79-8d35-419b-aa7d-ad627da991c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:28.210092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b76c79-8d35-419b-aa7d-ad627da991c4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:28.210405 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b76c79-8d35-419b-aa7d-ad627da991c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:28.211867 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b76c79-8d35-419b-aa7d-ad627da991c4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:28.222083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 179/362] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:51:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:28.222825 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b76c79-8d35-419b-aa7d-ad627da991c4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:28.231048 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b7195-9528-45dd-90cb-89b6a46d9292 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:28.269109 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:28.276784 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d303f-6c67-4857-a719-da3e3fa2644d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:28.277429 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:28.277644 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d303f-6c67-4857-a719-da3e3fa2644d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:28.278022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d303f-6c67-4857-a719-da3e3fa2644d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:28.278392 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d303f-6c67-4857-a719-da3e3fa2644d None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:28.279198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d303f-6c67-4857-a719-da3e3fa2644d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:28.280659 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d303f-6c67-4857-a719-da3e3fa2644d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:28.281108 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d303f-6c67-4857-a719-da3e3fa2644d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:28.293714 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 184/363] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:51:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:28.295694 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d303f-6c67-4857-a719-da3e3fa2644d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:28.349789 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b76c79-8d35-419b-aa7d-ad627da991c4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:28.394462 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22f8a2-a975-4fea-be1c-d325f653d4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:28.395270 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22f8a2-a975-4fea-be1c-d325f653d4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:28.395731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22f8a2-a975-4fea-be1c-d325f653d4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:28.396009 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22f8a2-a975-4fea-be1c-d325f653d4a5 None admin] PATH_INFO: `/v3/groups/543cc929fec0446e956024ebce66e97b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:28.397143 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e22f8a2-a975-4fea-be1c-d325f653d4a5 None admin] RBAC: Authorizing `identity:get_group(group_id=543cc929fec0446e956024ebce66e97b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:28.398578 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e22f8a2-a975-4fea-be1c-d325f653d4a5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:28.400329 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 180/364] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 00:51:28 2022] GET /identity/v3/groups/543cc929fec0446e956024ebce66e97b => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:28.400986 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22f8a2-a975-4fea-be1c-d325f653d4a5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:28.410442 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d303f-6c67-4857-a719-da3e3fa2644d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:28.451769 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700a787-514d-472a-9b10-c9e9adb86ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:28.452777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700a787-514d-472a-9b10-c9e9adb86ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:28.453062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700a787-514d-472a-9b10-c9e9adb86ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:28.453592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700a787-514d-472a-9b10-c9e9adb86ec8 None admin] PATH_INFO: `/v3/groups/543cc929fec0446e956024ebce66e97b` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:28.454456 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4700a787-514d-472a-9b10-c9e9adb86ec8 None admin] RBAC: Authorizing `identity:get_group(group_id=543cc929fec0446e956024ebce66e97b)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:28.456727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4700a787-514d-472a-9b10-c9e9adb86ec8 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:28.458521 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 185/365] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 00:51:28 2022] GET /identity/v3/groups/543cc929fec0446e956024ebce66e97b => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:28.459066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700a787-514d-472a-9b10-c9e9adb86ec8 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:28.519637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22f8a2-a975-4fea-be1c-d325f653d4a5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:28.521398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:28.532305 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:28.554319 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb5326-c1c8-4419-b5da-71276c1861e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:28.554925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb5326-c1c8-4419-b5da-71276c1861e4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:28.555206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb5326-c1c8-4419-b5da-71276c1861e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:28.555475 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb5326-c1c8-4419-b5da-71276c1861e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:28.556225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb5326-c1c8-4419-b5da-71276c1861e4 None admin] RBAC: Adding query filter params (group.id=543cc929fec0446e956024ebce66e97b, role.id=5d9045e45840408a92ceca71e9184ba2, scope.project.id=99c39c8f77a74048b8069e2cf0f83ae6) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:28.556531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb5326-c1c8-4419-b5da-71276c1861e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:28.557973 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb5326-c1c8-4419-b5da-71276c1861e4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:28.561434 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700a787-514d-472a-9b10-c9e9adb86ec8 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:28.563357 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 181/366] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Sep 14 00:51:28 2022] GET /identity/v3/role_assignments?group.id=543cc929fec0446e956024ebce66e97b&scope.project.id=99c39c8f77a74048b8069e2cf0f83ae6&role.id=5d9045e45840408a92ceca71e9184ba2 => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:28.564213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb5326-c1c8-4419-b5da-71276c1861e4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:28.614459 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df01ea-5b43-4820-a6de-fd1dd0dd9a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:28.615148 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df01ea-5b43-4820-a6de-fd1dd0dd9a81 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:28.615769 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df01ea-5b43-4820-a6de-fd1dd0dd9a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:28.616058 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df01ea-5b43-4820-a6de-fd1dd0dd9a81 None admin] PATH_INFO: `/v3/users/e163c58a399d4ac4a2d2816c086a325c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:28.617503 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df01ea-5b43-4820-a6de-fd1dd0dd9a81 None admin] RBAC: Authorizing `identity:get_user(user_id=e163c58a399d4ac4a2d2816c086a325c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:28.619031 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df01ea-5b43-4820-a6de-fd1dd0dd9a81 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:28.620774 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 186/367] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:51:28 2022] GET /identity/v3/users/e163c58a399d4ac4a2d2816c086a325c => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:28.621601 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df01ea-5b43-4820-a6de-fd1dd0dd9a81 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:28.685900 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb5326-c1c8-4419-b5da-71276c1861e4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:28.725538 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c938996-079c-4627-873f-69d4d015b43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:28.726166 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c938996-079c-4627-873f-69d4d015b43b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:28.726449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c938996-079c-4627-873f-69d4d015b43b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:28.726723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c938996-079c-4627-873f-69d4d015b43b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:28.747015 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df01ea-5b43-4820-a6de-fd1dd0dd9a81 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:28.778984 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:28.784810 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0a2e2-6e27-4f29-9737-994e5c890f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:28.785318 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:28.785474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0a2e2-6e27-4f29-9737-994e5c890f80 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:28.785884 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0a2e2-6e27-4f29-9737-994e5c890f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:28.786223 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0a2e2-6e27-4f29-9737-994e5c890f80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:28.796880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc0a2e2-6e27-4f29-9737-994e5c890f80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:28.802874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-ebc0a2e2-6e27-4f29-9737-994e5c890f80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:51:28.802874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:28.802874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:28.802874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:28.802874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:28.802874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:28.802874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:28.802874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:28.802874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:28.802874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:28.802874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:28.802874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:28.802874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:28.802874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:28.802874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:28.802874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:28.803992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:28.803992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:28.803992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:28.803992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:28.803992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:28.803992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:28.803992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:28.803992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:28.803992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:28.803992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:28.803992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:28.803992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:28.803992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:28.803992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:28.803992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:28.803992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:28.804997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:28.804997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:28.804997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:28.804997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:28.804997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:28.804997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:28.804997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:28.804997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:28.804997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:28.804997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:28.804997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:51:28.804997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:28.804997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 187/368] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:51:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:28.804997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0a2e2-6e27-4f29-9737-994e5c890f80 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:28.820662 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c938996-079c-4627-873f-69d4d015b43b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:28.822123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c938996-079c-4627-873f-69d4d015b43b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:28.912883 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0a2e2-6e27-4f29-9737-994e5c890f80 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:28.940577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 182/369] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:51:28.940577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c938996-079c-4627-873f-69d4d015b43b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:28.953528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172e2f4-f7f3-49bc-8d76-ee7231ace787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:28.954227 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172e2f4-f7f3-49bc-8d76-ee7231ace787 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:28.954493 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172e2f4-f7f3-49bc-8d76-ee7231ace787 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:28.955170 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172e2f4-f7f3-49bc-8d76-ee7231ace787 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:28.955764 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3172e2f4-f7f3-49bc-8d76-ee7231ace787 None admin] RBAC: Adding query filter params (group.id=543cc929fec0446e956024ebce66e97b, role.id=5d9045e45840408a92ceca71e9184ba2, scope.project.id=ac5d363867c44c349f5be02f57cd5958) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:28.956073 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3172e2f4-f7f3-49bc-8d76-ee7231ace787 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:28.957539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3172e2f4-f7f3-49bc-8d76-ee7231ace787 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:28.963126 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 188/370] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Sep 14 00:51:28 2022] GET /identity/v3/role_assignments?group.id=543cc929fec0446e956024ebce66e97b&scope.project.id=ac5d363867c44c349f5be02f57cd5958&role.id=5d9045e45840408a92ceca71e9184ba2 => generated 260 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:28.968570 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172e2f4-f7f3-49bc-8d76-ee7231ace787 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:29.032566 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:29.060190 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:29.062569 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c938996-079c-4627-873f-69d4d015b43b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:29.102937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f75422-2a68-4b98-8d3b-74e554c97cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:29.103581 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f75422-2a68-4b98-8d3b-74e554c97cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:29.103897 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f75422-2a68-4b98-8d3b-74e554c97cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:29.104674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f75422-2a68-4b98-8d3b-74e554c97cf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:29.105007 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f75422-2a68-4b98-8d3b-74e554c97cf7 None admin] RBAC: Adding query filter params (group.id=543cc929fec0446e956024ebce66e97b, role.id=89a3bc954b0a45bfa40a16db0ad2e51c, scope.project.id=ac5d363867c44c349f5be02f57cd5958) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:29.105326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f75422-2a68-4b98-8d3b-74e554c97cf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:29.106822 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f75422-2a68-4b98-8d3b-74e554c97cf7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:29.114609 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 183/371] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Sep 14 00:51:28 2022] GET /identity/v3/role_assignments?group.id=543cc929fec0446e956024ebce66e97b&scope.project.id=ac5d363867c44c349f5be02f57cd5958&role.id=89a3bc954b0a45bfa40a16db0ad2e51c => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:29.115463 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f75422-2a68-4b98-8d3b-74e554c97cf7 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:29.121401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172e2f4-f7f3-49bc-8d76-ee7231ace787 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:29.172120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ac93a-aebb-4fb4-a79c-f911d68eefb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:29.172762 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ac93a-aebb-4fb4-a79c-f911d68eefb1 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:29.173047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ac93a-aebb-4fb4-a79c-f911d68eefb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:29.173895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ac93a-aebb-4fb4-a79c-f911d68eefb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:29.174653 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ac93a-aebb-4fb4-a79c-f911d68eefb1 None admin] RBAC: Adding query filter params (role.id=f330ff772cca47c4b89a406784e00800, user.id=e163c58a399d4ac4a2d2816c086a325c, scope.system=all) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:29.174965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ac93a-aebb-4fb4-a79c-f911d68eefb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:29.180720 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819ac93a-aebb-4fb4-a79c-f911d68eefb1 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:29.205521 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 189/372] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Sep 14 00:51:28 2022] GET /identity/v3/role_assignments?user.id=e163c58a399d4ac4a2d2816c086a325c&scope.system=all&role.id=f330ff772cca47c4b89a406784e00800 => generated 510 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:29.207642 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ac93a-aebb-4fb4-a79c-f911d68eefb1 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:29.325261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f75422-2a68-4b98-8d3b-74e554c97cf7 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:29.396095 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b19f4c-ad2d-46a7-a05f-4d198b3ae678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:29.397107 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b19f4c-ad2d-46a7-a05f-4d198b3ae678 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:29.397600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b19f4c-ad2d-46a7-a05f-4d198b3ae678 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:29.398079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b19f4c-ad2d-46a7-a05f-4d198b3ae678 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:29.399006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b19f4c-ad2d-46a7-a05f-4d198b3ae678 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:29.399701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b19f4c-ad2d-46a7-a05f-4d198b3ae678 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:29.403197 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b19f4c-ad2d-46a7-a05f-4d198b3ae678 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:29.418449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 184/373] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:29.430134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b19f4c-ad2d-46a7-a05f-4d198b3ae678 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:29.436562 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:29.463427 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ac93a-aebb-4fb4-a79c-f911d68eefb1 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:29.492978 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:29.574119 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b19f4c-ad2d-46a7-a05f-4d198b3ae678 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:29.596697 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c41ed-7e94-4105-b5f0-4a0e212420b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:29.597901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c41ed-7e94-4105-b5f0-4a0e212420b1 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:29.598396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c41ed-7e94-4105-b5f0-4a0e212420b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:29.598880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c41ed-7e94-4105-b5f0-4a0e212420b1 None admin] PATH_INFO: `/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:29.604122 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c41ed-7e94-4105-b5f0-4a0e212420b1 None admin] RBAC: Authorizing `identity:get_project(project_id=99c39c8f77a74048b8069e2cf0f83ae6)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:29.605984 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c41ed-7e94-4105-b5f0-4a0e212420b1 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:29.608086 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 190/374] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:29 2022] GET /identity/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6 => generated 302 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:29.618522 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4fa2d7-bb7a-464d-a937-df4733b71dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:29.619225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fa2d7-bb7a-464d-a937-df4733b71dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:29.619510 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fa2d7-bb7a-464d-a937-df4733b71dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:29.619805 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fa2d7-bb7a-464d-a937-df4733b71dc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:29.633595 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c41ed-7e94-4105-b5f0-4a0e212420b1 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:29.698308 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4fa2d7-bb7a-464d-a937-df4733b71dc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:29.699807 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4fa2d7-bb7a-464d-a937-df4733b71dc7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:29.715925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:29.813851 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 185/375] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:51:29.829470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4fa2d7-bb7a-464d-a937-df4733b71dc7 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:29.856527 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c41ed-7e94-4105-b5f0-4a0e212420b1 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:29.896259 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dcc04a-73bb-4ad4-9116-55a2ae5016d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:29.896907 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dcc04a-73bb-4ad4-9116-55a2ae5016d1 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:29.897193 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dcc04a-73bb-4ad4-9116-55a2ae5016d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:29.897463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dcc04a-73bb-4ad4-9116-55a2ae5016d1 None admin] PATH_INFO: `/v3/groups/543cc929fec0446e956024ebce66e97b` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:29.901467 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dcc04a-73bb-4ad4-9116-55a2ae5016d1 None admin] RBAC: Authorizing `identity:get_group(group_id=543cc929fec0446e956024ebce66e97b)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:29.902921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dcc04a-73bb-4ad4-9116-55a2ae5016d1 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:29.905693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 191/376] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 00:51:29 2022] GET /identity/v3/groups/543cc929fec0446e956024ebce66e97b => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:29.912246 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dcc04a-73bb-4ad4-9116-55a2ae5016d1 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:30.007464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:30.018425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4fa2d7-bb7a-464d-a937-df4733b71dc7 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:30.031904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:30.096118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd20471-9ec3-4e83-a4dc-8584dd0cc56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:30.096760 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd20471-9ec3-4e83-a4dc-8584dd0cc56c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:30.097040 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd20471-9ec3-4e83-a4dc-8584dd0cc56c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:30.097312 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd20471-9ec3-4e83-a4dc-8584dd0cc56c None admin] PATH_INFO: `/v3/projects/724534edd35a41c585d8ed624130e96a` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:30.100422 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd20471-9ec3-4e83-a4dc-8584dd0cc56c None admin] RBAC: Authorizing `identity:get_project(project_id=724534edd35a41c585d8ed624130e96a)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:30.101868 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd20471-9ec3-4e83-a4dc-8584dd0cc56c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:30.103506 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 186/377] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:29 2022] GET /identity/v3/projects/724534edd35a41c585d8ed624130e96a => generated 344 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:30.109582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd20471-9ec3-4e83-a4dc-8584dd0cc56c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:30.113836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dcc04a-73bb-4ad4-9116-55a2ae5016d1 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:30.184858 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a7996-c0fe-4e41-ab91-c4f2f84902b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:30.185472 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a7996-c0fe-4e41-ab91-c4f2f84902b9 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:30.185749 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a7996-c0fe-4e41-ab91-c4f2f84902b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:30.186019 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a7996-c0fe-4e41-ab91-c4f2f84902b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:30.186744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a7996-c0fe-4e41-ab91-c4f2f84902b9 None admin] RBAC: Adding query filter params (group.id=543cc929fec0446e956024ebce66e97b, role.id=89a3bc954b0a45bfa40a16db0ad2e51c, scope.project.id=99c39c8f77a74048b8069e2cf0f83ae6) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:30.187053 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a7996-c0fe-4e41-ab91-c4f2f84902b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:30.188511 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a7996-c0fe-4e41-ab91-c4f2f84902b9 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:30.200359 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 192/378] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Sep 14 00:51:29 2022] GET /identity/v3/role_assignments?group.id=543cc929fec0446e956024ebce66e97b&scope.project.id=99c39c8f77a74048b8069e2cf0f83ae6&role.id=89a3bc954b0a45bfa40a16db0ad2e51c => generated 260 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:30.291944 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd20471-9ec3-4e83-a4dc-8584dd0cc56c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:30.339615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696525e-0369-4cad-a33b-17dc46e5016a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:30.340261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696525e-0369-4cad-a33b-17dc46e5016a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:30.340540 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696525e-0369-4cad-a33b-17dc46e5016a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:30.340875 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696525e-0369-4cad-a33b-17dc46e5016a None admin] PATH_INFO: `/v3/groups/2a73abbcac8242eda6288e5276ad5015` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:30.351274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5696525e-0369-4cad-a33b-17dc46e5016a None admin] RBAC: Authorizing `identity:get_group(group_id=2a73abbcac8242eda6288e5276ad5015)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:30.352826 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5696525e-0369-4cad-a33b-17dc46e5016a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:30.354765 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 187/379] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 00:51:30 2022] GET /identity/v3/groups/2a73abbcac8242eda6288e5276ad5015 => generated 229 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:30.360660 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a7996-c0fe-4e41-ab91-c4f2f84902b9 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:30.489005 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a7996-c0fe-4e41-ab91-c4f2f84902b9 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:30.503809 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:30.527004 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd46be-184c-4c72-843a-bedf19069602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:30.527648 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd46be-184c-4c72-843a-bedf19069602 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:30.527931 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd46be-184c-4c72-843a-bedf19069602 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:30.528200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd46be-184c-4c72-843a-bedf19069602 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:30.528989 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd46be-184c-4c72-843a-bedf19069602 None admin] RBAC: Adding query filter params (group.id=2a73abbcac8242eda6288e5276ad5015, role.id=ada268ea63354a0297070369dfe3c6cc, scope.project.id=724534edd35a41c585d8ed624130e96a) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:30.529294 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd46be-184c-4c72-843a-bedf19069602 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:30.530788 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd46be-184c-4c72-843a-bedf19069602 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:30.537959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 193/380] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Sep 14 00:51:30 2022] GET /identity/v3/role_assignments?group.id=2a73abbcac8242eda6288e5276ad5015&scope.project.id=724534edd35a41c585d8ed624130e96a&role.id=ada268ea63354a0297070369dfe3c6cc => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:31.086509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f24ac5-ac32-4dfc-95e7-bd929bc5ca95 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:31.087050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f24ac5-ac32-4dfc-95e7-bd929bc5ca95 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:31.087050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f24ac5-ac32-4dfc-95e7-bd929bc5ca95 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:31.088703 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 188/381] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:31.094159 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d403287-2dd0-4554-b549-a75af28124e0 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:31.094575 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d403287-2dd0-4554-b549-a75af28124e0 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:31.094766 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d403287-2dd0-4554-b549-a75af28124e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:31.117298 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-5d403287-2dd0-4554-b549-a75af28124e0 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:31.118507 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5d403287-2dd0-4554-b549-a75af28124e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:31.191573 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 194/382] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:31.657476 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f24ac5-ac32-4dfc-95e7-bd929bc5ca95 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:31.701071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:31.713464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e621b3f-a02d-48f8-acd8-dc1fe21d1d13 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:31.714085 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e621b3f-a02d-48f8-acd8-dc1fe21d1d13 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:31.714572 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e621b3f-a02d-48f8-acd8-dc1fe21d1d13 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:31.716418 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 195/383] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:31.728750 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba51c1-52c2-4daf-93f4-0368babdf7c0 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:31.729333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba51c1-52c2-4daf-93f4-0368babdf7c0 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:31.729677 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba51c1-52c2-4daf-93f4-0368babdf7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:31.763787 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-eaba51c1-52c2-4daf-93f4-0368babdf7c0 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:31.766278 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-eaba51c1-52c2-4daf-93f4-0368babdf7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:31.797813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f24ac5-ac32-4dfc-95e7-bd929bc5ca95 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:31.855062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b0e3f-983d-4eaa-b78e-2eb48b59ef98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:31.855737 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b0e3f-983d-4eaa-b78e-2eb48b59ef98 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:31.856018 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b0e3f-983d-4eaa-b78e-2eb48b59ef98 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:31.856287 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b0e3f-983d-4eaa-b78e-2eb48b59ef98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:31.888557 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 196/384] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:51:31.889054 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b0e3f-983d-4eaa-b78e-2eb48b59ef98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:31.896033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-5a1b0e3f-983d-4eaa-b78e-2eb48b59ef98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:31.896033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:31.896033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:31.896033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:31.896033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:31.896033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:31.896033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:31.896033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:31.896033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:31.896033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:31.896033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:31.896033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:31.896033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:31.896033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:31.896033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:31.896033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:31.896958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:31.896958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:31.896958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:31.896958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:31.896958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:31.896958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:31.896958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:31.896958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:31.896958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:31.896958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:31.896958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:31.896958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:31.896958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:31.896958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:31.896958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:31.896958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:31.897801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:31.897801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:31.897801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:31.897801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:31.897801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:31.897801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:31.897801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:31.897801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:31.897801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:31.897801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:31.897801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:31.897801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:31.903952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 189/385] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:31.910505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba51c1-52c2-4daf-93f4-0368babdf7c0 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:31.972624 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:32.059631 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba51c1-52c2-4daf-93f4-0368babdf7c0 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:32.106874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01242211-2201-4eb5-89fb-762d14dc9ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:32.107494 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01242211-2201-4eb5-89fb-762d14dc9ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:32.107804 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01242211-2201-4eb5-89fb-762d14dc9ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:32.108078 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01242211-2201-4eb5-89fb-762d14dc9ee0 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:32.108749 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01242211-2201-4eb5-89fb-762d14dc9ee0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:32.109050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01242211-2201-4eb5-89fb-762d14dc9ee0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:32.110421 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01242211-2201-4eb5-89fb-762d14dc9ee0 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:32.120002 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 197/386] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:51:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:32.125768 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b0e3f-983d-4eaa-b78e-2eb48b59ef98 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:32.212005 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01242211-2201-4eb5-89fb-762d14dc9ee0 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:32.279081 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:32.292891 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b0e3f-983d-4eaa-b78e-2eb48b59ef98 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:32.337842 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690ce36-3fab-4e73-85e0-11019277d29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:32.338397 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690ce36-3fab-4e73-85e0-11019277d29f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:32.338674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690ce36-3fab-4e73-85e0-11019277d29f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:32.338945 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690ce36-3fab-4e73-85e0-11019277d29f None admin] PATH_INFO: `/v3/groups/543cc929fec0446e956024ebce66e97b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:32.340656 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c690ce36-3fab-4e73-85e0-11019277d29f None admin] RBAC: Authorizing `identity:get_group(group_id=543cc929fec0446e956024ebce66e97b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:32.342284 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c690ce36-3fab-4e73-85e0-11019277d29f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:32.347670 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 190/387] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 00:51:32 2022] GET /identity/v3/groups/543cc929fec0446e956024ebce66e97b => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:32.347838 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:32.347920 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf867af-a474-4f9c-ae24-5fc47b96afa7 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:32.347920 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf867af-a474-4f9c-ae24-5fc47b96afa7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:32.347920 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf867af-a474-4f9c-ae24-5fc47b96afa7 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:32.349365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 191/388] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:32.352776 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf867af-a474-4f9c-ae24-5fc47b96afa7 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:32.356255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01242211-2201-4eb5-89fb-762d14dc9ee0 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:32.401125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a486c8-c1dd-431b-987b-c8eae78bf19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:32.401817 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a486c8-c1dd-431b-987b-c8eae78bf19d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:32.402094 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a486c8-c1dd-431b-987b-c8eae78bf19d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:32.402416 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a486c8-c1dd-431b-987b-c8eae78bf19d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:32.415994 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a486c8-c1dd-431b-987b-c8eae78bf19d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:32.423378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-e8a486c8-c1dd-431b-987b-c8eae78bf19d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:32.423378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:32.423378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:32.423378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:32.423378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:32.423378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:32.423378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:32.423378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:32.423378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:32.423378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:32.423378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:32.423378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:32.423378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:32.423378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:32.423378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:32.423378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:32.424387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:32.424387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:32.424387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:32.424387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:32.424387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:32.424387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:32.424387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:32.424387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:32.424387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:32.424387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:32.424387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:32.424387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:32.424387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:32.424387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:32.424387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:32.424387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:32.425182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:32.425182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:32.425182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:32.425182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:32.425182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:32.425182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:32.425182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:32.425182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:32.425182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:32.425182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:32.425182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:32.425182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:32.425182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 198/389] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:51:32.425895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e31d09-4969-4f84-abde-a5327f27025a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:32.426199 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e31d09-4969-4f84-abde-a5327f27025a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:32.426481 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e31d09-4969-4f84-abde-a5327f27025a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:32.452114 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e3e31d09-4969-4f84-abde-a5327f27025a None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:32.453309 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e3e31d09-4969-4f84-abde-a5327f27025a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:32.475798 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf867af-a474-4f9c-ae24-5fc47b96afa7 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:32.515446 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1c48d-4639-4964-bd98-eb581c309be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:32.516760 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1c48d-4639-4964-bd98-eb581c309be8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:32.517386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1c48d-4639-4964-bd98-eb581c309be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:32.517958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1c48d-4639-4964-bd98-eb581c309be8 None admin] PATH_INFO: `/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:32.519331 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b1c48d-4639-4964-bd98-eb581c309be8 None admin] RBAC: Authorizing `identity:get_project(project_id=99c39c8f77a74048b8069e2cf0f83ae6)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:32.521025 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b1c48d-4639-4964-bd98-eb581c309be8 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:32.527750 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 192/390] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:32 2022] GET /identity/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:32.528834 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1c48d-4639-4964-bd98-eb581c309be8 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:32.546017 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 199/391] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:32.548062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e31d09-4969-4f84-abde-a5327f27025a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:32.583755 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:32.674776 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1c48d-4639-4964-bd98-eb581c309be8 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:32.709197 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:32.713638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbffd2c9-9f1d-4222-8ec7-26da38a01a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:32.714556 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffd2c9-9f1d-4222-8ec7-26da38a01a51 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:32.715755 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffd2c9-9f1d-4222-8ec7-26da38a01a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:32.716229 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffd2c9-9f1d-4222-8ec7-26da38a01a51 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:32.717082 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbffd2c9-9f1d-4222-8ec7-26da38a01a51 None admin] RBAC: Adding query filter params (name=member) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:32.717637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbffd2c9-9f1d-4222-8ec7-26da38a01a51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:32.719236 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbffd2c9-9f1d-4222-8ec7-26da38a01a51 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:32.730426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 193/392] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:51:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:32.738060 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e31d09-4969-4f84-abde-a5327f27025a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:32.741134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbffd2c9-9f1d-4222-8ec7-26da38a01a51 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:32.804252 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a1d0b-cab8-4964-8faf-62ad9cfbcea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:32.804931 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a1d0b-cab8-4964-8faf-62ad9cfbcea6 None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:32.805206 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a1d0b-cab8-4964-8faf-62ad9cfbcea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:32.805478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a1d0b-cab8-4964-8faf-62ad9cfbcea6 None admin] PATH_INFO: `/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6/groups/543cc929fec0446e956024ebce66e97b/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:32.810753 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a1d0b-cab8-4964-8faf-62ad9cfbcea6 None admin] RBAC: Authorizing `identity:create_grant(project_id=99c39c8f77a74048b8069e2cf0f83ae6, group_id=543cc929fec0446e956024ebce66e97b, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:32.812227 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a1d0b-cab8-4964-8faf-62ad9cfbcea6 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:32.839271 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 200/393] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Sep 14 00:51:32 2022] PUT /identity/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6/groups/543cc929fec0446e956024ebce66e97b/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:51:32.863109 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:32.875154 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbffd2c9-9f1d-4222-8ec7-26da38a01a51 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:32.914809 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c85ac-65c5-4d9e-a373-e12871094360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:32.915461 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c85ac-65c5-4d9e-a373-e12871094360 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:32.915769 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c85ac-65c5-4d9e-a373-e12871094360 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:32.916046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c85ac-65c5-4d9e-a373-e12871094360 None admin] PATH_INFO: `/v3/groups/543cc929fec0446e956024ebce66e97b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:32.917224 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c85ac-65c5-4d9e-a373-e12871094360 None admin] RBAC: Authorizing `identity:get_group(group_id=543cc929fec0446e956024ebce66e97b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:32.918773 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c85ac-65c5-4d9e-a373-e12871094360 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:32.922295 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656010a-7e57-4df6-a2f6-7146624d54a6 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:32.922583 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656010a-7e57-4df6-a2f6-7146624d54a6 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:32.922860 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656010a-7e57-4df6-a2f6-7146624d54a6 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:32.924561 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 194/394] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 00:51:32 2022] GET /identity/v3/groups/543cc929fec0446e956024ebce66e97b => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:32.924684 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 201/395] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:51:32.926952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c85ac-65c5-4d9e-a373-e12871094360 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:32.933183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef79314-70c2-4941-ab69-31b013cb3a9a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:32.933500 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef79314-70c2-4941-ab69-31b013cb3a9a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:32.933725 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef79314-70c2-4941-ab69-31b013cb3a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:32.952705 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-6ef79314-70c2-4941-ab69-31b013cb3a9a None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:32.953958 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6ef79314-70c2-4941-ab69-31b013cb3a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:33.071133 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c85ac-65c5-4d9e-a373-e12871094360 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:33.078049 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 202/396] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:33.078906 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef79314-70c2-4941-ab69-31b013cb3a9a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:33.117443 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091af0e-478f-4971-a44d-987668f7e446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:33.117906 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091af0e-478f-4971-a44d-987668f7e446 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:33.118405 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091af0e-478f-4971-a44d-987668f7e446 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:33.118849 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091af0e-478f-4971-a44d-987668f7e446 None admin] PATH_INFO: `/v3/projects/ac5d363867c44c349f5be02f57cd5958` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:33.120784 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091af0e-478f-4971-a44d-987668f7e446 None admin] RBAC: Authorizing `identity:get_project(project_id=ac5d363867c44c349f5be02f57cd5958)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:33.123084 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091af0e-478f-4971-a44d-987668f7e446 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:33.125242 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 195/397] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:32 2022] GET /identity/v3/projects/ac5d363867c44c349f5be02f57cd5958 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:33.131533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091af0e-478f-4971-a44d-987668f7e446 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:33.169028 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:33.182699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:33.215156 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef79314-70c2-4941-ab69-31b013cb3a9a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:33.260612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091af0e-478f-4971-a44d-987668f7e446 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:33.263777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d698972-0331-42a6-9b19-bd32666932d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:33.268377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d698972-0331-42a6-9b19-bd32666932d8 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:33.269327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d698972-0331-42a6-9b19-bd32666932d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:33.269790 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d698972-0331-42a6-9b19-bd32666932d8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:33.285167 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d698972-0331-42a6-9b19-bd32666932d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:33.292629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-5d698972-0331-42a6-9b19-bd32666932d8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:51:33.292629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:33.292629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:33.292629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:33.292629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:33.292629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:33.292629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:33.292629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:33.292629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:33.292629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:33.292629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:33.292629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:33.292629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:33.292629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:33.292629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:33.292629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:33.293820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:33.293820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:33.293820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:33.293820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:33.293820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:33.293820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:33.293820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:33.293820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:33.293820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:33.293820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:33.293820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:33.293820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:33.293820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:33.293820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:33.293820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:33.293820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:33.294815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:33.294815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:33.294815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:33.294815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:33.294815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:33.294815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:33.294815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:33.294815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:33.294815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:33.294815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:33.294815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:51:33.294815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:33.302428 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ad17e-6b49-4f58-a435-b57d8f63860c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:33.302428 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ad17e-6b49-4f58-a435-b57d8f63860c None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:33.302428 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ad17e-6b49-4f58-a435-b57d8f63860c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:33.302428 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ad17e-6b49-4f58-a435-b57d8f63860c None admin] PATH_INFO: `/v3/projects/ac5d363867c44c349f5be02f57cd5958/groups/543cc929fec0446e956024ebce66e97b/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:33.302873 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 203/398] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 00:51:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:33.304529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ad17e-6b49-4f58-a435-b57d8f63860c None admin] RBAC: Authorizing `identity:create_grant(project_id=ac5d363867c44c349f5be02f57cd5958, group_id=543cc929fec0446e956024ebce66e97b, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:33.305038 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab516a-43dd-4c31-ae6d-a63c5cb2d212 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:33.305483 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab516a-43dd-4c31-ae6d-a63c5cb2d212 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:33.305955 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab516a-43dd-4c31-ae6d-a63c5cb2d212 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:33.306332 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ad17e-6b49-4f58-a435-b57d8f63860c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:33.307825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 204/399] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:33.308904 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab516a-43dd-4c31-ae6d-a63c5cb2d212 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:33.326182 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 196/400] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Sep 14 00:51:33 2022] PUT /identity/v3/projects/ac5d363867c44c349f5be02f57cd5958/groups/543cc929fec0446e956024ebce66e97b/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:51:33.327908 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d3f987-ec95-4219-8d0e-24795a558229 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:33.328783 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d3f987-ec95-4219-8d0e-24795a558229 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:33.329181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d3f987-ec95-4219-8d0e-24795a558229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:33.351962 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b4d3f987-ec95-4219-8d0e-24795a558229 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:33.353327 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b4d3f987-ec95-4219-8d0e-24795a558229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:33.436023 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 197/401] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:33.437296 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d3f987-ec95-4219-8d0e-24795a558229 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:33.453152 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab516a-43dd-4c31-ae6d-a63c5cb2d212 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:33.473215 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:33.541335 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d6816b-e651-42cd-befd-68c8016e41a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:33.542409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6816b-e651-42cd-befd-68c8016e41a1 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:33.542697 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6816b-e651-42cd-befd-68c8016e41a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:33.546757 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6816b-e651-42cd-befd-68c8016e41a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:33.550516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d6816b-e651-42cd-befd-68c8016e41a1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:33.550830 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d6816b-e651-42cd-befd-68c8016e41a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:33.552302 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d6816b-e651-42cd-befd-68c8016e41a1 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:33.565639 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:33.571015 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 205/402] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:51:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:33.576094 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d6816b-e651-42cd-befd-68c8016e41a1 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:33.596235 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d3f987-ec95-4219-8d0e-24795a558229 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:33.637421 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a1f4a-d713-42ce-b988-0649daddf673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:33.638393 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a1f4a-d713-42ce-b988-0649daddf673 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:33.638898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a1f4a-d713-42ce-b988-0649daddf673 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:33.639315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a1f4a-d713-42ce-b988-0649daddf673 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:33.653833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a1f4a-d713-42ce-b988-0649daddf673 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:33.660564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-848a1f4a-d713-42ce-b988-0649daddf673 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:51:33.660564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:33.660564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:33.660564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:33.660564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:33.660564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:33.660564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:33.660564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:33.660564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:33.660564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:33.660564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:33.660564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:33.660564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:33.660564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:33.660564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:33.660564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:33.661645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:33.661645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:33.661645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:33.661645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:33.661645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:33.661645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:33.661645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:33.661645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:33.661645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:33.661645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:33.661645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:33.661645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:33.661645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:33.661645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:33.661645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:33.661645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:33.662521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:33.662521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:33.662521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:33.662521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:33.662521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:33.662521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:33.662521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:33.662521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:33.662521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:33.662521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:33.662521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:51:33.662521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:33.669546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 198/403] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 00:51:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:33.671016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a1f4a-d713-42ce-b988-0649daddf673 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:33.699878 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d6816b-e651-42cd-befd-68c8016e41a1 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:33.736251 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ccbc9-acce-40a9-a903-51be9c5c0987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:33.737014 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ccbc9-acce-40a9-a903-51be9c5c0987 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:33.737539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ccbc9-acce-40a9-a903-51be9c5c0987 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:33.737866 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ccbc9-acce-40a9-a903-51be9c5c0987 None admin] PATH_INFO: `/v3/groups/543cc929fec0446e956024ebce66e97b` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:33.739126 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ccbc9-acce-40a9-a903-51be9c5c0987 None admin] RBAC: Authorizing `identity:get_group(group_id=543cc929fec0446e956024ebce66e97b)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:33.740606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ccbc9-acce-40a9-a903-51be9c5c0987 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:33.742508 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 206/404] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 00:51:33 2022] GET /identity/v3/groups/543cc929fec0446e956024ebce66e97b => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:33.747511 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ccbc9-acce-40a9-a903-51be9c5c0987 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:33.817962 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:33.830887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:33.846344 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a1f4a-d713-42ce-b988-0649daddf673 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:33.874612 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ccbc9-acce-40a9-a903-51be9c5c0987 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:33.885225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b82447-63c2-4922-92f3-65098c44fee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:33.886163 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b82447-63c2-4922-92f3-65098c44fee2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:33.886587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b82447-63c2-4922-92f3-65098c44fee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:33.887118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b82447-63c2-4922-92f3-65098c44fee2 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:33.887952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b82447-63c2-4922-92f3-65098c44fee2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:33.888421 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b82447-63c2-4922-92f3-65098c44fee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:33.890049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b82447-63c2-4922-92f3-65098c44fee2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:33.900189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 199/405] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:51:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:33.901291 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b82447-63c2-4922-92f3-65098c44fee2 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:33.917116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a44dd2-1b2b-46da-b866-f5600ee985f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:33.917855 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a44dd2-1b2b-46da-b866-f5600ee985f8 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:33.918144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a44dd2-1b2b-46da-b866-f5600ee985f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:33.918721 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a44dd2-1b2b-46da-b866-f5600ee985f8 None admin] PATH_INFO: `/v3/projects/ac5d363867c44c349f5be02f57cd5958` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:33.920051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a44dd2-1b2b-46da-b866-f5600ee985f8 None admin] RBAC: Authorizing `identity:get_project(project_id=ac5d363867c44c349f5be02f57cd5958)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:33.922178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a44dd2-1b2b-46da-b866-f5600ee985f8 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:33.929035 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 207/406] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:33 2022] GET /identity/v3/projects/ac5d363867c44c349f5be02f57cd5958 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:33.929035 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a44dd2-1b2b-46da-b866-f5600ee985f8 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:34.024676 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b82447-63c2-4922-92f3-65098c44fee2 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:34.057007 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a44dd2-1b2b-46da-b866-f5600ee985f8 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:34.073665 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee7644-3940-4bf6-9f62-01f29ad7742d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:34.074394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee7644-3940-4bf6-9f62-01f29ad7742d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:34.074688 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee7644-3940-4bf6-9f62-01f29ad7742d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:34.074965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee7644-3940-4bf6-9f62-01f29ad7742d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:34.085964 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:34.092834 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee7644-3940-4bf6-9f62-01f29ad7742d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:34.097288 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a68a00-2721-40e3-9778-9e88f29d36ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:34.098181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a68a00-2721-40e3-9778-9e88f29d36ea None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:34.098659 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a68a00-2721-40e3-9778-9e88f29d36ea None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:34.098764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-94ee7644-3940-4bf6-9f62-01f29ad7742d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:51:34.098764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:34.098764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:34.098764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:34.098764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:34.098764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:34.098764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:34.098764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:34.098764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:34.098764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:34.098764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:34.098764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:34.098764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:34.098764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:34.098764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:34.098764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:34.099614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:34.099614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:34.099614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:34.099614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:34.099614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:34.099614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:34.099614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:34.099614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:34.099614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:34.099614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:34.099614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:34.099614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:34.099614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:34.099614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:34.099614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:34.099614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:34.100570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:34.100570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:34.100570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:34.100570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:34.100570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:34.100570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:34.100570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:34.100570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:34.100570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:34.100570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:34.100570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:51:34.100570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:34.100570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 200/407] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:51:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:34.106412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee7644-3940-4bf6-9f62-01f29ad7742d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:34.107010 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a68a00-2721-40e3-9778-9e88f29d36ea None admin] PATH_INFO: `/v3/groups/543cc929fec0446e956024ebce66e97b` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:34.108432 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a68a00-2721-40e3-9778-9e88f29d36ea None admin] RBAC: Authorizing `identity:get_group(group_id=543cc929fec0446e956024ebce66e97b)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:34.110064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a68a00-2721-40e3-9778-9e88f29d36ea None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:34.112288 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 208/408] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 00:51:33 2022] GET /identity/v3/groups/543cc929fec0446e956024ebce66e97b => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:34.113252 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a68a00-2721-40e3-9778-9e88f29d36ea None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:34.118250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:34.231314 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a68a00-2721-40e3-9778-9e88f29d36ea None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:34.249941 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee7644-3940-4bf6-9f62-01f29ad7742d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:34.268994 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ddafa-263a-4df7-bb18-defebf892fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:34.269616 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ddafa-263a-4df7-bb18-defebf892fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:34.269900 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ddafa-263a-4df7-bb18-defebf892fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:34.270177 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ddafa-263a-4df7-bb18-defebf892fa7 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:34.271058 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ddafa-263a-4df7-bb18-defebf892fa7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:34.271166 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ddafa-263a-4df7-bb18-defebf892fa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:34.272530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ddafa-263a-4df7-bb18-defebf892fa7 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:34.286813 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 209/409] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:34.287677 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ddafa-263a-4df7-bb18-defebf892fa7 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:34.295974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34807025-78c2-4ce0-a28c-2e2b1e5c8ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:34.296928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34807025-78c2-4ce0-a28c-2e2b1e5c8ad6 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:34.297849 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34807025-78c2-4ce0-a28c-2e2b1e5c8ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:34.298413 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34807025-78c2-4ce0-a28c-2e2b1e5c8ad6 None admin] PATH_INFO: `/v3/projects/ac5d363867c44c349f5be02f57cd5958/groups/543cc929fec0446e956024ebce66e97b/roles/89a3bc954b0a45bfa40a16db0ad2e51c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:34.301182 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34807025-78c2-4ce0-a28c-2e2b1e5c8ad6 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac5d363867c44c349f5be02f57cd5958, group_id=543cc929fec0446e956024ebce66e97b, role_id=89a3bc954b0a45bfa40a16db0ad2e51c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:34.303423 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34807025-78c2-4ce0-a28c-2e2b1e5c8ad6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:34.322168 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 201/410] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Sep 14 00:51:34 2022] PUT /identity/v3/projects/ac5d363867c44c349f5be02f57cd5958/groups/543cc929fec0446e956024ebce66e97b/roles/89a3bc954b0a45bfa40a16db0ad2e51c => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:51:34.323428 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34807025-78c2-4ce0-a28c-2e2b1e5c8ad6 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:34.404271 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:34.412000 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:34.450284 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34807025-78c2-4ce0-a28c-2e2b1e5c8ad6 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:34.493726 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013a0ce-5163-46f6-bee3-3467ab2da813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:34.494703 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013a0ce-5163-46f6-bee3-3467ab2da813 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:34.495224 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013a0ce-5163-46f6-bee3-3467ab2da813 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:34.495746 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013a0ce-5163-46f6-bee3-3467ab2da813 None admin] PATH_INFO: `/v3/groups/2a73abbcac8242eda6288e5276ad5015` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:34.497330 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013a0ce-5163-46f6-bee3-3467ab2da813 None admin] RBAC: Authorizing `identity:get_group(group_id=2a73abbcac8242eda6288e5276ad5015)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:34.499199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013a0ce-5163-46f6-bee3-3467ab2da813 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:34.501449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 202/411] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 00:51:34 2022] GET /identity/v3/groups/2a73abbcac8242eda6288e5276ad5015 => generated 229 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:34.507865 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013a0ce-5163-46f6-bee3-3467ab2da813 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:34.515428 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ddafa-263a-4df7-bb18-defebf892fa7 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:34.634388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013a0ce-5163-46f6-bee3-3467ab2da813 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:34.641628 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322ae77-98c2-4e2c-ad8b-3ad974aa3c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:34.642880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322ae77-98c2-4e2c-ad8b-3ad974aa3c67 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:34.643165 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322ae77-98c2-4e2c-ad8b-3ad974aa3c67 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:34.643438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322ae77-98c2-4e2c-ad8b-3ad974aa3c67 None admin] PATH_INFO: `/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:34.644632 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f322ae77-98c2-4e2c-ad8b-3ad974aa3c67 None admin] RBAC: Authorizing `identity:get_project(project_id=99c39c8f77a74048b8069e2cf0f83ae6)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:34.646096 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f322ae77-98c2-4e2c-ad8b-3ad974aa3c67 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:34.648978 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 210/412] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:34 2022] GET /identity/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6 => generated 302 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:34.656535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322ae77-98c2-4e2c-ad8b-3ad974aa3c67 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:34.669984 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:34.681627 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b4f7f-e027-4a94-aa32-2477cffbb6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:34.682527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b4f7f-e027-4a94-aa32-2477cffbb6db None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:34.683013 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b4f7f-e027-4a94-aa32-2477cffbb6db None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:34.683531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b4f7f-e027-4a94-aa32-2477cffbb6db None admin] PATH_INFO: `/v3/projects/724534edd35a41c585d8ed624130e96a` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:34.685469 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b4f7f-e027-4a94-aa32-2477cffbb6db None admin] RBAC: Authorizing `identity:get_project(project_id=724534edd35a41c585d8ed624130e96a)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:34.687687 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b4f7f-e027-4a94-aa32-2477cffbb6db None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:34.691385 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 203/413] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:34 2022] GET /identity/v3/projects/724534edd35a41c585d8ed624130e96a => generated 344 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:34.696921 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b4f7f-e027-4a94-aa32-2477cffbb6db None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:34.779745 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322ae77-98c2-4e2c-ad8b-3ad974aa3c67 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:34.819501 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeae709-de0a-4771-b829-6a3e7625239d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:34.820315 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeae709-de0a-4771-b829-6a3e7625239d None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:34.820926 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeae709-de0a-4771-b829-6a3e7625239d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:34.820926 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeae709-de0a-4771-b829-6a3e7625239d None admin] PATH_INFO: `/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6/groups/543cc929fec0446e956024ebce66e97b/roles/89a3bc954b0a45bfa40a16db0ad2e51c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:34.823387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeae709-de0a-4771-b829-6a3e7625239d None admin] RBAC: Authorizing `identity:create_grant(project_id=99c39c8f77a74048b8069e2cf0f83ae6, group_id=543cc929fec0446e956024ebce66e97b, role_id=89a3bc954b0a45bfa40a16db0ad2e51c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:34.825235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeae709-de0a-4771-b829-6a3e7625239d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:34.827487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:34.837375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b4f7f-e027-4a94-aa32-2477cffbb6db None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:34.850632 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 211/414] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Sep 14 00:51:34 2022] PUT /identity/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6/groups/543cc929fec0446e956024ebce66e97b/roles/89a3bc954b0a45bfa40a16db0ad2e51c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:51:34.886558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c55b4a-89db-42df-a69f-4afd4e587997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:34.887681 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c55b4a-89db-42df-a69f-4afd4e587997 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:34.888196 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c55b4a-89db-42df-a69f-4afd4e587997 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:34.888677 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c55b4a-89db-42df-a69f-4afd4e587997 None admin] PATH_INFO: `/v3/projects/724534edd35a41c585d8ed624130e96a/groups/2a73abbcac8242eda6288e5276ad5015/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:34.897265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c55b4a-89db-42df-a69f-4afd4e587997 None admin] RBAC: Authorizing `identity:create_grant(project_id=724534edd35a41c585d8ed624130e96a, group_id=2a73abbcac8242eda6288e5276ad5015, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:34.899114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c55b4a-89db-42df-a69f-4afd4e587997 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:34.926992 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 204/415] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Sep 14 00:51:34 2022] PUT /identity/v3/projects/724534edd35a41c585d8ed624130e96a/groups/2a73abbcac8242eda6288e5276ad5015/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:51:35.292830 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0af75-4cd2-436e-9912-fe4e2d592e99 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:35.293865 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0af75-4cd2-436e-9912-fe4e2d592e99 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:35.294382 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0af75-4cd2-436e-9912-fe4e2d592e99 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:35.295979 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 212/416] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:35.302273 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd005b-83a8-4f92-bf0b-a1513f8741bf None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:35.302703 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd005b-83a8-4f92-bf0b-a1513f8741bf None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:35.302894 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd005b-83a8-4f92-bf0b-a1513f8741bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:35.332605 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-28dd005b-83a8-4f92-bf0b-a1513f8741bf None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:35.333825 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-28dd005b-83a8-4f92-bf0b-a1513f8741bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:35.410654 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:35.433306 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 205/417] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:35.818070 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0af75-4cd2-436e-9912-fe4e2d592e99 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:35.961031 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0af75-4cd2-436e-9912-fe4e2d592e99 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:36.010428 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758c36a-56d4-4534-9884-4bd706bcbfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:36.011472 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758c36a-56d4-4534-9884-4bd706bcbfb0 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:36.011990 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758c36a-56d4-4534-9884-4bd706bcbfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:36.012481 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758c36a-56d4-4534-9884-4bd706bcbfb0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:36.018505 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eead11-b9bd-4d5d-a4fa-c1922e33b610 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:36.018842 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eead11-b9bd-4d5d-a4fa-c1922e33b610 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:36.019060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eead11-b9bd-4d5d-a4fa-c1922e33b610 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:36.020807 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 206/418] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:36.027826 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758c36a-56d4-4534-9884-4bd706bcbfb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:36.034402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-c758c36a-56d4-4534-9884-4bd706bcbfb0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:36.034402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:36.034402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:36.034402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:36.034402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:36.034402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:36.034402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:36.034402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:36.034402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:36.034402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:36.034402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:36.034402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:36.034402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:36.034402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:36.034402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:36.034402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:36.035347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:36.035347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:36.035347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:36.035347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:36.035347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:36.035347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:36.035347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:36.035347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:36.035347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:36.035347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:36.035347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:36.035347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:36.035347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:36.035347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:36.035347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:36.035347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:36.036196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:36.036196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:36.036196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:36.036196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:36.036196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:36.036196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:36.036196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:36.036196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:36.036196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:36.036196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:36.036196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:36.036196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:36.036801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3360946-28b8-440a-a011-774081e14427 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:36.036801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3360946-28b8-440a-a011-774081e14427 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:36.036801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3360946-28b8-440a-a011-774081e14427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:36.039053 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 213/419] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:36.044324 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758c36a-56d4-4534-9884-4bd706bcbfb0 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:36.067489 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f3360946-28b8-440a-a011-774081e14427 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:36.069223 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f3360946-28b8-440a-a011-774081e14427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:36.145579 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 207/420] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:36.160232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:36.227121 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758c36a-56d4-4534-9884-4bd706bcbfb0 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:36.277133 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbe408-c85e-4934-9939-7f1733c04a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:36.277799 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbe408-c85e-4934-9939-7f1733c04a35 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:36.278084 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbe408-c85e-4934-9939-7f1733c04a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:36.278360 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbe408-c85e-4934-9939-7f1733c04a35 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:36.279032 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dbe408-c85e-4934-9939-7f1733c04a35 None admin] RBAC: Adding query filter params (name=member) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:36.279337 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dbe408-c85e-4934-9939-7f1733c04a35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:36.280744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dbe408-c85e-4934-9939-7f1733c04a35 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:36.291966 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 214/421] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:51:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:36.297867 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3360946-28b8-440a-a011-774081e14427 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:36.433890 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3360946-28b8-440a-a011-774081e14427 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:36.456313 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:36.484669 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590d8f1d-d34d-4525-8644-ab03daa36f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:36.485305 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d8f1d-d34d-4525-8644-ab03daa36f46 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:36.485637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d8f1d-d34d-4525-8644-ab03daa36f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:36.485920 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d8f1d-d34d-4525-8644-ab03daa36f46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:36.499256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbe408-c85e-4934-9939-7f1733c04a35 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:36.608128 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590d8f1d-d34d-4525-8644-ab03daa36f46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:36.609640 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590d8f1d-d34d-4525-8644-ab03daa36f46 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:36.638710 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbe408-c85e-4934-9939-7f1733c04a35 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:36.670391 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:36.675893 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64536a9b-c671-413b-baa3-1ed1f36eb127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:36.676529 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64536a9b-c671-413b-baa3-1ed1f36eb127 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:36.676807 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64536a9b-c671-413b-baa3-1ed1f36eb127 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:36.677081 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64536a9b-c671-413b-baa3-1ed1f36eb127 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:36.688124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64536a9b-c671-413b-baa3-1ed1f36eb127 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:36.693805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-64536a9b-c671-413b-baa3-1ed1f36eb127 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:36.693805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:36.693805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:36.693805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:36.693805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:36.693805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:36.693805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:36.693805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:36.693805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:36.693805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:36.693805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:36.693805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:36.693805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:36.693805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:36.693805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:36.693805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:36.694902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:36.694902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:36.694902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:36.694902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:36.694902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:36.694902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:36.694902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:36.694902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:36.694902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:36.694902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:36.694902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:36.694902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:36.694902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:36.694902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:36.694902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:36.694902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:36.695925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:36.695925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:36.695925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:36.695925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:36.695925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:36.695925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:36.695925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:36.695925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:36.695925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:36.695925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:36.695925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 00:51:36.695925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:36.698226 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 215/422] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:36.703697 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64536a9b-c671-413b-baa3-1ed1f36eb127 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:36.840346 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 208/423] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:51:36.846287 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590d8f1d-d34d-4525-8644-ab03daa36f46 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:36.877548 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:36.892826 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64536a9b-c671-413b-baa3-1ed1f36eb127 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:36.985193 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590d8f1d-d34d-4525-8644-ab03daa36f46 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:37.008286 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64532fe0-debd-4dec-8ffe-7f839a1e0b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:37.008924 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64532fe0-debd-4dec-8ffe-7f839a1e0b7c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:37.009211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64532fe0-debd-4dec-8ffe-7f839a1e0b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:37.009576 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64532fe0-debd-4dec-8ffe-7f839a1e0b7c None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:37.010159 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64532fe0-debd-4dec-8ffe-7f839a1e0b7c None admin] RBAC: Adding query filter params (name=member) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:37.010465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64532fe0-debd-4dec-8ffe-7f839a1e0b7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:37.011904 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64532fe0-debd-4dec-8ffe-7f839a1e0b7c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:37.023224 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd1a5d-fdd9-473c-9a8e-52a994941201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:37.024206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd1a5d-fdd9-473c-9a8e-52a994941201 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:37.024636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd1a5d-fdd9-473c-9a8e-52a994941201 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:37.025069 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd1a5d-fdd9-473c-9a8e-52a994941201 None admin] PATH_INFO: `/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:37.026377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd1a5d-fdd9-473c-9a8e-52a994941201 None admin] RBAC: Authorizing `identity:get_project(project_id=99c39c8f77a74048b8069e2cf0f83ae6)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:37.028046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd1a5d-fdd9-473c-9a8e-52a994941201 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:37.030392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 209/424] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:36 2022] GET /identity/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:37.036349 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd1a5d-fdd9-473c-9a8e-52a994941201 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:37.038280 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 216/425] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:51:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:37.043966 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64532fe0-debd-4dec-8ffe-7f839a1e0b7c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:37.112848 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:37.149093 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:37.187295 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd1a5d-fdd9-473c-9a8e-52a994941201 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:37.238553 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731af8b4-8f9e-423a-b658-54bc3c35302f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:37.239625 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731af8b4-8f9e-423a-b658-54bc3c35302f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:37.240130 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731af8b4-8f9e-423a-b658-54bc3c35302f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:37.240606 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731af8b4-8f9e-423a-b658-54bc3c35302f None admin] PATH_INFO: `/v3/groups/543cc929fec0446e956024ebce66e97b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:37.241929 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731af8b4-8f9e-423a-b658-54bc3c35302f None admin] RBAC: Authorizing `identity:get_group(group_id=543cc929fec0446e956024ebce66e97b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:37.243984 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731af8b4-8f9e-423a-b658-54bc3c35302f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:37.246070 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 210/426] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 00:51:37 2022] GET /identity/v3/groups/543cc929fec0446e956024ebce66e97b => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:37.253257 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731af8b4-8f9e-423a-b658-54bc3c35302f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:37.261332 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64532fe0-debd-4dec-8ffe-7f839a1e0b7c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:37.311778 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fbd137-7d22-47f3-a769-7d38d096ccf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:37.313020 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbd137-7d22-47f3-a769-7d38d096ccf5 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:37.314287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbd137-7d22-47f3-a769-7d38d096ccf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:37.314793 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbd137-7d22-47f3-a769-7d38d096ccf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:37.375626 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731af8b4-8f9e-423a-b658-54bc3c35302f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:37.401195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fbd137-7d22-47f3-a769-7d38d096ccf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:37.402673 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fbd137-7d22-47f3-a769-7d38d096ccf5 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:37.425316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47451348-7800-4465-8182-f00d98960ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:37.427048 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47451348-7800-4465-8182-f00d98960ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:37.427891 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47451348-7800-4465-8182-f00d98960ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:37.429475 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47451348-7800-4465-8182-f00d98960ed7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:37.430256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47451348-7800-4465-8182-f00d98960ed7 None admin] RBAC: Adding query filter params (group.id=543cc929fec0446e956024ebce66e97b, role.id=5d9045e45840408a92ceca71e9184ba2, scope.project.id=99c39c8f77a74048b8069e2cf0f83ae6) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:37.430570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47451348-7800-4465-8182-f00d98960ed7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:37.432095 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47451348-7800-4465-8182-f00d98960ed7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:37.443918 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 211/427] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Sep 14 00:51:37 2022] GET /identity/v3/role_assignments?group.id=543cc929fec0446e956024ebce66e97b&scope.project.id=99c39c8f77a74048b8069e2cf0f83ae6&role.id=5d9045e45840408a92ceca71e9184ba2 => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:37.443918 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1fa85-64f7-48d8-8fab-afe54332a006 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:37.443918 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1fa85-64f7-48d8-8fab-afe54332a006 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:37.443918 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1fa85-64f7-48d8-8fab-afe54332a006 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:37.444754 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 212/428] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:37.449279 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68df8d5-b4d6-411b-9dd7-6c729c438c05 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:37.449569 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68df8d5-b4d6-411b-9dd7-6c729c438c05 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:37.449850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68df8d5-b4d6-411b-9dd7-6c729c438c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:37.470559 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:37.484238 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a68df8d5-b4d6-411b-9dd7-6c729c438c05 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:37.486028 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a68df8d5-b4d6-411b-9dd7-6c729c438c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:37.512652 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:37.589543 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 217/429] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 14 00:51:37.591224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4639fd-332a-44d1-98b0-d8212ccb606a None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:37.591431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4639fd-332a-44d1-98b0-d8212ccb606a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:37.591730 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4639fd-332a-44d1-98b0-d8212ccb606a None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:37.593739 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 218/430] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:37.597499 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4639fd-332a-44d1-98b0-d8212ccb606a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:37.607948 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 213/431] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:51:37.609313 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f7e56-3942-44c2-9d11-571b48e9ccfb None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:37.609528 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f7e56-3942-44c2-9d11-571b48e9ccfb None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:37.609798 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f7e56-3942-44c2-9d11-571b48e9ccfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:37.631238 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-112f7e56-3942-44c2-9d11-571b48e9ccfb None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:37.632503 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-112f7e56-3942-44c2-9d11-571b48e9ccfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:37.710383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 214/432] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:37.712137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd24c22-f29a-4098-8770-fd04444b3ccc None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:37.712630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd24c22-f29a-4098-8770-fd04444b3ccc None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:37.713877 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd24c22-f29a-4098-8770-fd04444b3ccc None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:37.716288 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 215/433] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:37.722928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a44606-8794-4947-aa04-b7d01097ec09 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:37.723494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a44606-8794-4947-aa04-b7d01097ec09 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:37.724005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a44606-8794-4947-aa04-b7d01097ec09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:37.733961 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4639fd-332a-44d1-98b0-d8212ccb606a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:37.744359 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-74a44606-8794-4947-aa04-b7d01097ec09 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:37.745962 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-74a44606-8794-4947-aa04-b7d01097ec09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:37.773802 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867536d-3882-4b2f-b798-e54204f97047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:37.774594 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867536d-3882-4b2f-b798-e54204f97047 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:37.774940 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867536d-3882-4b2f-b798-e54204f97047 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:37.775284 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867536d-3882-4b2f-b798-e54204f97047 None admin] PATH_INFO: `/v3/projects/ac5d363867c44c349f5be02f57cd5958` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:37.776535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867536d-3882-4b2f-b798-e54204f97047 None admin] RBAC: Authorizing `identity:get_project(project_id=ac5d363867c44c349f5be02f57cd5958)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:37.778233 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867536d-3882-4b2f-b798-e54204f97047 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:37.785388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 219/434] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:37 2022] GET /identity/v3/projects/ac5d363867c44c349f5be02f57cd5958 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:37.786521 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867536d-3882-4b2f-b798-e54204f97047 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:37.798969 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:37.844841 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 216/435] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:37.911059 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867536d-3882-4b2f-b798-e54204f97047 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:37.943632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a44606-8794-4947-aa04-b7d01097ec09 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:37.951759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21173307-ba1a-47d9-9c42-9cb43ab98bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:37.952766 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:37.954105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21173307-ba1a-47d9-9c42-9cb43ab98bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:37.954919 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21173307-ba1a-47d9-9c42-9cb43ab98bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:37.956351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21173307-ba1a-47d9-9c42-9cb43ab98bd3 None admin] PATH_INFO: `/v3/groups/543cc929fec0446e956024ebce66e97b` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:37.959212 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21173307-ba1a-47d9-9c42-9cb43ab98bd3 None admin] RBAC: Authorizing `identity:get_group(group_id=543cc929fec0446e956024ebce66e97b)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:37.961627 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21173307-ba1a-47d9-9c42-9cb43ab98bd3 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:37.964303 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 220/436] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 00:51:37 2022] GET /identity/v3/groups/543cc929fec0446e956024ebce66e97b => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:37.970429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21173307-ba1a-47d9-9c42-9cb43ab98bd3 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:38.053534 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:38.068313 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a44606-8794-4947-aa04-b7d01097ec09 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:38.079439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21173307-ba1a-47d9-9c42-9cb43ab98bd3 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:38.111001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bd13f-7d5f-4dbc-ab8f-dc1b37d74cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:38.111676 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bd13f-7d5f-4dbc-ab8f-dc1b37d74cdc None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:38.111961 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bd13f-7d5f-4dbc-ab8f-dc1b37d74cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:38.112236 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bd13f-7d5f-4dbc-ab8f-dc1b37d74cdc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:38.114766 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa760b-aab2-4ab3-a77c-1093d65da679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:38.115365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa760b-aab2-4ab3-a77c-1093d65da679 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:38.115674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa760b-aab2-4ab3-a77c-1093d65da679 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:38.115945 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa760b-aab2-4ab3-a77c-1093d65da679 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:38.116664 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa760b-aab2-4ab3-a77c-1093d65da679 None admin] RBAC: Adding query filter params (group.id=543cc929fec0446e956024ebce66e97b, role.id=5d9045e45840408a92ceca71e9184ba2, scope.project.id=ac5d363867c44c349f5be02f57cd5958) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:38.116968 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa760b-aab2-4ab3-a77c-1093d65da679 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:38.118402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa760b-aab2-4ab3-a77c-1093d65da679 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:38.123678 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450bd13f-7d5f-4dbc-ab8f-dc1b37d74cdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:38.130887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 221/437] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Sep 14 00:51:37 2022] GET /identity/v3/role_assignments?group.id=543cc929fec0446e956024ebce66e97b&scope.project.id=ac5d363867c44c349f5be02f57cd5958&role.id=5d9045e45840408a92ceca71e9184ba2 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:38.130887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa760b-aab2-4ab3-a77c-1093d65da679 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:38.131151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-450bd13f-7d5f-4dbc-ab8f-dc1b37d74cdc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:51:38.131151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:38.131151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:38.131151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:38.131151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:38.131151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:38.131151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:38.131151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:38.131151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:38.131151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:38.131151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:38.131151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:38.131151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:38.131151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:38.131151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:38.131151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:38.132557 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:38.132557 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:38.132557 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:38.132557 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:38.132557 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:38.132557 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:38.132557 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:38.132557 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:38.132557 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:38.132557 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:38.132557 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:38.132557 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:38.132557 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:38.132557 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:38.132557 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:38.132557 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:38.133350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:38.133350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:38.133350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:38.133350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:38.133350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:38.133350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:38.133350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:38.133350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:38.133350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:38.133350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:38.133350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:51:38.133350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:38.133350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 217/438] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 00:51:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:51:38.133350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bd13f-7d5f-4dbc-ab8f-dc1b37d74cdc None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:38.214042 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:38.246490 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa760b-aab2-4ab3-a77c-1093d65da679 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:38.253313 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bd13f-7d5f-4dbc-ab8f-dc1b37d74cdc None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:38.288178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d31f2-ec43-4a0d-b875-d834949c1492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:38.288879 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d31f2-ec43-4a0d-b875-d834949c1492 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:38.289170 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d31f2-ec43-4a0d-b875-d834949c1492 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:38.289690 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d31f2-ec43-4a0d-b875-d834949c1492 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:38.294552 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cf785-1ae3-4e5c-8dfe-82b037a0a756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:38.295202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cf785-1ae3-4e5c-8dfe-82b037a0a756 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:38.295477 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cf785-1ae3-4e5c-8dfe-82b037a0a756 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:38.295775 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cf785-1ae3-4e5c-8dfe-82b037a0a756 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:38.297787 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:38.303067 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d31f2-ec43-4a0d-b875-d834949c1492 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:38.307351 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49cf785-1ae3-4e5c-8dfe-82b037a0a756 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:38.310959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-705d31f2-ec43-4a0d-b875-d834949c1492 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:51:38.310959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:38.310959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:38.310959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:38.310959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:38.310959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:38.310959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:38.310959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:38.310959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:38.310959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:38.310959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:38.310959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:38.310959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:38.310959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:38.310959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:38.310959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:38.311880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:38.311880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:38.311880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:38.311880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:38.311880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:38.311880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:38.311880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:38.311880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:38.311880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:38.311880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:38.311880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:38.311880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:38.311880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:38.311880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:38.311880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:38.311880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:38.313128 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:38.313128 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:38.313128 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:38.313128 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:38.313128 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:38.313128 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:38.313128 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:38.313128 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:38.313128 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:38.313128 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:38.313128 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:51:38.313128 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:38.314255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-f49cf785-1ae3-4e5c-8dfe-82b037a0a756 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:51:38.314255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:38.314255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:38.314255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:38.314255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:38.314255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:38.314255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:38.314255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:38.314255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:38.314255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:38.314255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:38.314255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:38.314255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:38.314255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:38.314255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:38.314255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:38.315419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:38.315419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:38.315419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:38.315419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:38.315419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:38.315419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:38.315419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:38.315419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:38.315419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:38.315419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:38.315419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:38.315419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:38.315419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:38.315419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:38.315419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:38.315419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:38.316418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:38.316418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:38.316418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:38.316418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:38.316418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:38.316418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:38.316418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:38.316418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:38.316418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:38.316418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:38.316418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:51:38.316418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:38.317169 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 222/439] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 00:51:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:38.317169 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d31f2-ec43-4a0d-b875-d834949c1492 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:38.317329 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 218/440] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:51:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:51:38.320315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cf785-1ae3-4e5c-8dfe-82b037a0a756 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:38.432746 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d31f2-ec43-4a0d-b875-d834949c1492 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:38.440636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cf785-1ae3-4e5c-8dfe-82b037a0a756 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:38.470601 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:38.473869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1422a0f-5db0-4901-a189-99f2b5556aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:38.474631 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1422a0f-5db0-4901-a189-99f2b5556aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:38.475079 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1422a0f-5db0-4901-a189-99f2b5556aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:38.475363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1422a0f-5db0-4901-a189-99f2b5556aa5 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:38.477002 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1422a0f-5db0-4901-a189-99f2b5556aa5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:38.477002 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1422a0f-5db0-4901-a189-99f2b5556aa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:38.477771 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1422a0f-5db0-4901-a189-99f2b5556aa5 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:38.484530 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175eb123-554f-42b7-9af2-0bf9711aceb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:38.485158 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175eb123-554f-42b7-9af2-0bf9711aceb2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:38.485441 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175eb123-554f-42b7-9af2-0bf9711aceb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:38.485782 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175eb123-554f-42b7-9af2-0bf9711aceb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:38.486462 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175eb123-554f-42b7-9af2-0bf9711aceb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:38.487480 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175eb123-554f-42b7-9af2-0bf9711aceb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:38.487933 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 223/441] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:51:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:38.489290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1422a0f-5db0-4901-a189-99f2b5556aa5 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:38.492390 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175eb123-554f-42b7-9af2-0bf9711aceb2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:38.501793 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 219/442] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:38.503278 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175eb123-554f-42b7-9af2-0bf9711aceb2 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:38.560689 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:38.601837 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1422a0f-5db0-4901-a189-99f2b5556aa5 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:38.611799 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175eb123-554f-42b7-9af2-0bf9711aceb2 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:38.634509 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814f2d3-2b51-4e64-866f-a7a81b0c5ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:38.635114 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814f2d3-2b51-4e64-866f-a7a81b0c5ffe None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:38.635398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814f2d3-2b51-4e64-866f-a7a81b0c5ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:38.635709 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814f2d3-2b51-4e64-866f-a7a81b0c5ffe None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:38.636377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814f2d3-2b51-4e64-866f-a7a81b0c5ffe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:38.636684 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814f2d3-2b51-4e64-866f-a7a81b0c5ffe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:38.638058 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814f2d3-2b51-4e64-866f-a7a81b0c5ffe None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:38.643963 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee2694-c732-4385-8e4c-d3d89902b568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:38.644565 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee2694-c732-4385-8e4c-d3d89902b568 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:38.644839 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee2694-c732-4385-8e4c-d3d89902b568 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:38.645106 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee2694-c732-4385-8e4c-d3d89902b568 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:38.646178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 224/443] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:51:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:38.646971 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814f2d3-2b51-4e64-866f-a7a81b0c5ffe None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:38.703378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:38.716657 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee2694-c732-4385-8e4c-d3d89902b568 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:38.718097 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee2694-c732-4385-8e4c-d3d89902b568 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:38.744503 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814f2d3-2b51-4e64-866f-a7a81b0c5ffe None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:38.766474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:38.778786 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c47477-61ec-4ff8-ac09-bc3e5ef1aa4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:38.779478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c47477-61ec-4ff8-ac09-bc3e5ef1aa4f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:38.779847 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c47477-61ec-4ff8-ac09-bc3e5ef1aa4f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:38.780183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c47477-61ec-4ff8-ac09-bc3e5ef1aa4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:38.816202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 220/444] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:51:38.816982 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee2694-c732-4385-8e4c-d3d89902b568 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:38.849125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c47477-61ec-4ff8-ac09-bc3e5ef1aa4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:38.850582 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c47477-61ec-4ff8-ac09-bc3e5ef1aa4f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:38.902625 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:38.914694 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee2694-c732-4385-8e4c-d3d89902b568 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:38.948788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f023fb-aee4-41ed-82b5-2e23768f10b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:38.949393 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f023fb-aee4-41ed-82b5-2e23768f10b5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:38.949680 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f023fb-aee4-41ed-82b5-2e23768f10b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:38.949950 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f023fb-aee4-41ed-82b5-2e23768f10b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:38.950479 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 225/445] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:51:38.951312 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c47477-61ec-4ff8-ac09-bc3e5ef1aa4f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:38.976076 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:39.018391 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f023fb-aee4-41ed-82b5-2e23768f10b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:39.019889 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f023fb-aee4-41ed-82b5-2e23768f10b5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:39.051280 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c47477-61ec-4ff8-ac09-bc3e5ef1aa4f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:39.086048 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac8a17d-bf61-454b-b623-daec67501020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:39.086629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8a17d-bf61-454b-b623-daec67501020 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:39.086908 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8a17d-bf61-454b-b623-daec67501020 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:39.087180 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8a17d-bf61-454b-b623-daec67501020 None admin] PATH_INFO: `/v3/projects/ac5d363867c44c349f5be02f57cd5958` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:39.088323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac8a17d-bf61-454b-b623-daec67501020 None admin] RBAC: Authorizing `identity:get_project(project_id=ac5d363867c44c349f5be02f57cd5958)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:39.089770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac8a17d-bf61-454b-b623-daec67501020 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:39.091742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 226/446] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:38 2022] GET /identity/v3/projects/ac5d363867c44c349f5be02f57cd5958 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:39.092346 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac8a17d-bf61-454b-b623-daec67501020 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:39.108581 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:39.122253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 221/447] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:51:39.123078 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f023fb-aee4-41ed-82b5-2e23768f10b5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:39.184920 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:39.195838 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac8a17d-bf61-454b-b623-daec67501020 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:39.227611 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f023fb-aee4-41ed-82b5-2e23768f10b5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:39.229726 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cbb5e4-c1dc-4973-9229-698fec8ba9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:39.230359 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbb5e4-c1dc-4973-9229-698fec8ba9f0 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:39.230642 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbb5e4-c1dc-4973-9229-698fec8ba9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:39.230917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbb5e4-c1dc-4973-9229-698fec8ba9f0 None admin] PATH_INFO: `/v3/projects/724534edd35a41c585d8ed624130e96a` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:39.232187 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cbb5e4-c1dc-4973-9229-698fec8ba9f0 None admin] RBAC: Authorizing `identity:get_project(project_id=724534edd35a41c585d8ed624130e96a)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:39.233639 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cbb5e4-c1dc-4973-9229-698fec8ba9f0 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:39.235242 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 227/448] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:39 2022] GET /identity/v3/projects/724534edd35a41c585d8ed624130e96a => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:39.236114 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cbb5e4-c1dc-4973-9229-698fec8ba9f0 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:39.264209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2048927-64fa-4d60-b1e8-0b9d84a16da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:39.264838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2048927-64fa-4d60-b1e8-0b9d84a16da0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:39.265124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2048927-64fa-4d60-b1e8-0b9d84a16da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:39.265403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2048927-64fa-4d60-b1e8-0b9d84a16da0 None admin] PATH_INFO: `/v3/groups/543cc929fec0446e956024ebce66e97b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:39.267355 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2048927-64fa-4d60-b1e8-0b9d84a16da0 None admin] RBAC: Authorizing `identity:get_group(group_id=543cc929fec0446e956024ebce66e97b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:39.268823 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2048927-64fa-4d60-b1e8-0b9d84a16da0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:39.270376 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 222/449] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 00:51:39 2022] GET /identity/v3/groups/543cc929fec0446e956024ebce66e97b => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:39.271130 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2048927-64fa-4d60-b1e8-0b9d84a16da0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:39.326573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:39.335412 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cbb5e4-c1dc-4973-9229-698fec8ba9f0 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:39.368642 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3de9d-1435-4655-9ab7-b2de52b4c8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:39.369199 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3de9d-1435-4655-9ab7-b2de52b4c8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:39.369480 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3de9d-1435-4655-9ab7-b2de52b4c8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:39.369759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3de9d-1435-4655-9ab7-b2de52b4c8e9 None admin] PATH_INFO: `/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:39.369852 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2048927-64fa-4d60-b1e8-0b9d84a16da0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:39.370870 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3de9d-1435-4655-9ab7-b2de52b4c8e9 None admin] RBAC: Authorizing `identity:get_project(project_id=99c39c8f77a74048b8069e2cf0f83ae6)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:39.372373 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3de9d-1435-4655-9ab7-b2de52b4c8e9 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:39.373987 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 228/450] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:51:39 2022] GET /identity/v3/projects/99c39c8f77a74048b8069e2cf0f83ae6 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:39.374748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3de9d-1435-4655-9ab7-b2de52b4c8e9 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:39.401698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:39.407278 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5fee60-be1b-4a83-a874-79f720d40c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:39.408472 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5fee60-be1b-4a83-a874-79f720d40c1d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:39.408754 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5fee60-be1b-4a83-a874-79f720d40c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:39.409031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5fee60-be1b-4a83-a874-79f720d40c1d None admin] PATH_INFO: `/v3/groups/2a73abbcac8242eda6288e5276ad5015` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:39.410153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5fee60-be1b-4a83-a874-79f720d40c1d None admin] RBAC: Authorizing `identity:get_group(group_id=2a73abbcac8242eda6288e5276ad5015)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:39.411635 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5fee60-be1b-4a83-a874-79f720d40c1d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:39.413206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 223/451] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 00:51:39 2022] GET /identity/v3/groups/2a73abbcac8242eda6288e5276ad5015 => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:39.414006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5fee60-be1b-4a83-a874-79f720d40c1d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:39.475677 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3de9d-1435-4655-9ab7-b2de52b4c8e9 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:39.512209 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24c314-33c3-4bb2-8f0b-a47e793bbcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:39.512821 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24c314-33c3-4bb2-8f0b-a47e793bbcab None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:39.513096 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24c314-33c3-4bb2-8f0b-a47e793bbcab None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:39.513367 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24c314-33c3-4bb2-8f0b-a47e793bbcab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:39.514080 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec24c314-33c3-4bb2-8f0b-a47e793bbcab None admin] RBAC: Adding query filter params (group.id=543cc929fec0446e956024ebce66e97b, role.id=89a3bc954b0a45bfa40a16db0ad2e51c, scope.project.id=ac5d363867c44c349f5be02f57cd5958) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:39.514384 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec24c314-33c3-4bb2-8f0b-a47e793bbcab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:39.516065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec24c314-33c3-4bb2-8f0b-a47e793bbcab None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:39.518810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5fee60-be1b-4a83-a874-79f720d40c1d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:39.522437 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 229/452] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Sep 14 00:51:39 2022] GET /identity/v3/role_assignments?group.id=543cc929fec0446e956024ebce66e97b&scope.project.id=ac5d363867c44c349f5be02f57cd5958&role.id=89a3bc954b0a45bfa40a16db0ad2e51c => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:39.523224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24c314-33c3-4bb2-8f0b-a47e793bbcab None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:39.544000 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:39.558942 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42697e7e-746e-4c51-94f6-8457c302681a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:39.559733 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42697e7e-746e-4c51-94f6-8457c302681a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:39.560015 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42697e7e-746e-4c51-94f6-8457c302681a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:39.560309 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42697e7e-746e-4c51-94f6-8457c302681a None admin] PATH_INFO: `/v3/groups/543cc929fec0446e956024ebce66e97b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:39.561495 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42697e7e-746e-4c51-94f6-8457c302681a None admin] RBAC: Authorizing `identity:get_group(group_id=543cc929fec0446e956024ebce66e97b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:39.562977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42697e7e-746e-4c51-94f6-8457c302681a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:39.565777 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 224/453] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 00:51:39 2022] GET /identity/v3/groups/543cc929fec0446e956024ebce66e97b => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:39.571537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42697e7e-746e-4c51-94f6-8457c302681a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:39.631251 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:39.644044 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24c314-33c3-4bb2-8f0b-a47e793bbcab None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:39.683138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b813037-761f-4283-8b5f-5e8393a89795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:39.683811 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b813037-761f-4283-8b5f-5e8393a89795 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:39.684093 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b813037-761f-4283-8b5f-5e8393a89795 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:39.684372 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b813037-761f-4283-8b5f-5e8393a89795 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:39.685104 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b813037-761f-4283-8b5f-5e8393a89795 None admin] RBAC: Adding query filter params (group.id=2a73abbcac8242eda6288e5276ad5015, role.id=ada268ea63354a0297070369dfe3c6cc, scope.project.id=724534edd35a41c585d8ed624130e96a) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:39.685414 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b813037-761f-4283-8b5f-5e8393a89795 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:39.686907 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b813037-761f-4283-8b5f-5e8393a89795 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:39.691204 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42697e7e-746e-4c51-94f6-8457c302681a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:39.693531 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 230/454] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Sep 14 00:51:39 2022] GET /identity/v3/role_assignments?group.id=2a73abbcac8242eda6288e5276ad5015&scope.project.id=724534edd35a41c585d8ed624130e96a&role.id=ada268ea63354a0297070369dfe3c6cc => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:39.735179 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8848a0-a4d0-41d0-b67e-fa5297a209c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:39.736438 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8848a0-a4d0-41d0-b67e-fa5297a209c6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:39.736718 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8848a0-a4d0-41d0-b67e-fa5297a209c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:39.736987 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8848a0-a4d0-41d0-b67e-fa5297a209c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:39.737721 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8848a0-a4d0-41d0-b67e-fa5297a209c6 None admin] RBAC: Adding query filter params (group.id=543cc929fec0446e956024ebce66e97b, role.id=89a3bc954b0a45bfa40a16db0ad2e51c, scope.project.id=99c39c8f77a74048b8069e2cf0f83ae6) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:39.738026 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8848a0-a4d0-41d0-b67e-fa5297a209c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:39.739475 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8848a0-a4d0-41d0-b67e-fa5297a209c6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:39.746181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 225/455] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Sep 14 00:51:39 2022] GET /identity/v3/role_assignments?group.id=543cc929fec0446e956024ebce66e97b&scope.project.id=99c39c8f77a74048b8069e2cf0f83ae6&role.id=89a3bc954b0a45bfa40a16db0ad2e51c => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:42.691819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee24197-19eb-4bce-8665-0eaf65768990 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:42.693217 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee24197-19eb-4bce-8665-0eaf65768990 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:42.693709 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee24197-19eb-4bce-8665-0eaf65768990 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:42.695501 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 231/456] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:42.700859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21460b-f823-4f30-b320-6d8c72c5d743 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:42.701651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21460b-f823-4f30-b320-6d8c72c5d743 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:42.702160 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21460b-f823-4f30-b320-6d8c72c5d743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:42.732917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ad21460b-f823-4f30-b320-6d8c72c5d743 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:42.734436 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ad21460b-f823-4f30-b320-6d8c72c5d743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:42.808812 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:42.830152 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 226/457] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:42.965706 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af7d45-a23e-428e-8d78-5bee5288cc87 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:42.966229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af7d45-a23e-428e-8d78-5bee5288cc87 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:42.966676 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af7d45-a23e-428e-8d78-5bee5288cc87 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:42.968512 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 232/458] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:42.975337 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b1e2d-5d58-462d-a1a8-179a1757e1c2 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:42.975881 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b1e2d-5d58-462d-a1a8-179a1757e1c2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:42.976327 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b1e2d-5d58-462d-a1a8-179a1757e1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:43.008756 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-102b1e2d-5d58-462d-a1a8-179a1757e1c2 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:43.010183 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-102b1e2d-5d58-462d-a1a8-179a1757e1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:43.106111 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 227/459] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:51:43.201910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af7d45-a23e-428e-8d78-5bee5288cc87 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:43.205590 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b5a47-1a9a-45d1-b572-d3e2fded0fbb None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:43.205791 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b5a47-1a9a-45d1-b572-d3e2fded0fbb None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:43.206036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b5a47-1a9a-45d1-b572-d3e2fded0fbb None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:43.207874 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 228/460] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:51:43.212670 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300f00a-979f-48b8-ac2e-6fafa596fd54 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:43.213245 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300f00a-979f-48b8-ac2e-6fafa596fd54 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:43.213777 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300f00a-979f-48b8-ac2e-6fafa596fd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:43.234916 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0300f00a-979f-48b8-ac2e-6fafa596fd54 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:43.236958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0300f00a-979f-48b8-ac2e-6fafa596fd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:43.363459 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 229/461] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:51:43.389863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af7d45-a23e-428e-8d78-5bee5288cc87 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:43.432929 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55336ccf-d70e-4121-9049-c291be73767e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:43.433912 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55336ccf-d70e-4121-9049-c291be73767e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:43.434388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55336ccf-d70e-4121-9049-c291be73767e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:43.434855 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55336ccf-d70e-4121-9049-c291be73767e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:43.446502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55336ccf-d70e-4121-9049-c291be73767e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:43.448275 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55336ccf-d70e-4121-9049-c291be73767e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:43.465245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-55336ccf-d70e-4121-9049-c291be73767e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:43.465245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:43.465245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:43.465245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:43.465245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:43.465245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:43.465245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:43.465245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:43.465245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:43.465245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:43.465245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:43.465245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:43.465245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:51:43.465245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:51:43.465245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:51:43.465245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:51:43.466207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:43.466207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:43.466207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:43.466207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:43.466207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:43.466207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:43.466207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:43.466207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:43.466207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:43.466207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:43.466207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:43.466207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:43.466207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:43.466207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:43.466207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:43.466207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:43.466207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:51:43.467050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:51:43.467050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:43.467050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:43.468846 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 233/462] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:43.476988 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300f00a-979f-48b8-ac2e-6fafa596fd54 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:43.495355 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d6185-26c8-4d2f-bfb6-9ff9aa3cde51 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:43.495983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d6185-26c8-4d2f-bfb6-9ff9aa3cde51 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:43.496252 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d6185-26c8-4d2f-bfb6-9ff9aa3cde51 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:43.497714 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 234/463] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:51:43.499628 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d6185-26c8-4d2f-bfb6-9ff9aa3cde51 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:43.512579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:43.539115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:43.600684 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300f00a-979f-48b8-ac2e-6fafa596fd54 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:43.624802 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d6185-26c8-4d2f-bfb6-9ff9aa3cde51 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:43.641169 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80bbc00-8a7e-43a3-b2e6-5ba93fb37262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:43.643549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bbc00-8a7e-43a3-b2e6-5ba93fb37262 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:43.644034 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bbc00-8a7e-43a3-b2e6-5ba93fb37262 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:43.644763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bbc00-8a7e-43a3-b2e6-5ba93fb37262 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:43.645740 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80bbc00-8a7e-43a3-b2e6-5ba93fb37262 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:43.646249 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80bbc00-8a7e-43a3-b2e6-5ba93fb37262 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:43.647918 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80bbc00-8a7e-43a3-b2e6-5ba93fb37262 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:43.663319 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 230/464] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:43.664881 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a7e17-4b3a-4e93-bad6-2ee44dda9148 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:43.665467 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a7e17-4b3a-4e93-bad6-2ee44dda9148 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:43.665991 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a7e17-4b3a-4e93-bad6-2ee44dda9148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:43.670290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f1c06-4af6-4176-b9ee-d5610deea69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:43.670927 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f1c06-4af6-4176-b9ee-d5610deea69d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:43.671206 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f1c06-4af6-4176-b9ee-d5610deea69d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:43.671484 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f1c06-4af6-4176-b9ee-d5610deea69d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:43.685743 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f1c06-4af6-4176-b9ee-d5610deea69d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:43.687146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4c7a7e17-4b3a-4e93-bad6-2ee44dda9148 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:43.687381 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f1c06-4af6-4176-b9ee-d5610deea69d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:43.689310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4c7a7e17-4b3a-4e93-bad6-2ee44dda9148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:43.706321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-d27f1c06-4af6-4176-b9ee-d5610deea69d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:43.706321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:43.706321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:43.706321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:43.706321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:43.706321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:43.706321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:43.706321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:43.706321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:43.706321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:43.706321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:43.706321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:43.706321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:51:43.706321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:51:43.706321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:51:43.706321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:51:43.707614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:43.707614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:43.707614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:43.707614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:43.707614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:43.707614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:43.707614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:43.707614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:43.707614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:43.707614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:43.707614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:43.707614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:43.707614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:43.707614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:43.707614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:43.707614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:43.707614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:51:43.709089 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:51:43.709089 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:43.709089 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:43.709089 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 235/465] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:43.709089 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f1c06-4af6-4176-b9ee-d5610deea69d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:43.771121 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 231/466] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:43.776473 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a7e17-4b3a-4e93-bad6-2ee44dda9148 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:43.785238 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:43.821070 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:43.835459 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f1c06-4af6-4176-b9ee-d5610deea69d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:43.872344 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7b7ed-7fe8-4f90-89b7-415b5c64b01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:43.872954 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7b7ed-7fe8-4f90-89b7-415b5c64b01b None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:43.873233 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7b7ed-7fe8-4f90-89b7-415b5c64b01b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:43.873505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7b7ed-7fe8-4f90-89b7-415b5c64b01b None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:43.874153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb7b7ed-7fe8-4f90-89b7-415b5c64b01b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:43.875822 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb7b7ed-7fe8-4f90-89b7-415b5c64b01b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:43.907586 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 236/467] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 00:51:43 2022] POST /identity/v3/users => generated 246 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:51:43.907586 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7b7ed-7fe8-4f90-89b7-415b5c64b01b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:43.919937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a7e17-4b3a-4e93-bad6-2ee44dda9148 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:43.963440 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58aec2-7526-42ce-a7d2-ddaa4332b21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:43.964101 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58aec2-7526-42ce-a7d2-ddaa4332b21c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:43.964384 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58aec2-7526-42ce-a7d2-ddaa4332b21c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:43.964660 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58aec2-7526-42ce-a7d2-ddaa4332b21c None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:43.965322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58aec2-7526-42ce-a7d2-ddaa4332b21c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:43.965635 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58aec2-7526-42ce-a7d2-ddaa4332b21c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:43.967784 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58aec2-7526-42ce-a7d2-ddaa4332b21c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:43.984053 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 232/468] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:43.985572 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506542a-98e5-49c8-a34b-bd1e97e84897 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:43.985951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506542a-98e5-49c8-a34b-bd1e97e84897 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:43.986232 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506542a-98e5-49c8-a34b-bd1e97e84897 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:43.992176 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 233/469] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:43.993230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506542a-98e5-49c8-a34b-bd1e97e84897 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:44.034015 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7b7ed-7fe8-4f90-89b7-415b5c64b01b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:44.050096 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:44.074964 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48213971-61f8-4f4f-9a49-13a1d6b1ff77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:44.075645 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48213971-61f8-4f4f-9a49-13a1d6b1ff77 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:44.075931 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48213971-61f8-4f4f-9a49-13a1d6b1ff77 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:44.076205 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48213971-61f8-4f4f-9a49-13a1d6b1ff77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:44.088698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48213971-61f8-4f4f-9a49-13a1d6b1ff77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:44.090201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48213971-61f8-4f4f-9a49-13a1d6b1ff77 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:44.106077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-48213971-61f8-4f4f-9a49-13a1d6b1ff77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:44.106077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:44.106077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:44.106077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:44.106077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:44.106077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:44.106077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:44.106077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:44.106077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:44.106077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:44.106077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:44.106077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:44.106077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:51:44.106077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:51:44.106077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:51:44.106077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:51:44.107250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:44.107250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:44.107250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:44.107250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:44.107250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:44.107250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:44.107250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:44.107250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:44.107250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:44.107250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:44.107250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:44.107250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:44.107250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:44.107250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:44.107250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:44.107250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:44.107250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:51:44.108408 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:51:44.108408 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:44.108408 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:44.109845 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 237/470] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:44.111381 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a59138-161b-4645-94ec-f131f0bf10b8 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:44.111705 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a59138-161b-4645-94ec-f131f0bf10b8 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:44.111829 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:44.112035 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a59138-161b-4645-94ec-f131f0bf10b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:44.120500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506542a-98e5-49c8-a34b-bd1e97e84897 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:44.131787 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-32a59138-161b-4645-94ec-f131f0bf10b8 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:44.133214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-32a59138-161b-4645-94ec-f131f0bf10b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:44.163136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a6223-8cf7-4758-a4e7-2f225cc22c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:44.163796 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a6223-8cf7-4758-a4e7-2f225cc22c3f None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:44.164078 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a6223-8cf7-4758-a4e7-2f225cc22c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:44.164354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a6223-8cf7-4758-a4e7-2f225cc22c3f None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:44.165135 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a6223-8cf7-4758-a4e7-2f225cc22c3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:44.166711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a6223-8cf7-4758-a4e7-2f225cc22c3f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:44.190955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 234/471] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 00:51:43 2022] POST /identity/v3/users => generated 248 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:51:44.192662 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a6223-8cf7-4758-a4e7-2f225cc22c3f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:44.213805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 238/472] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:44.214834 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a59138-161b-4645-94ec-f131f0bf10b8 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:44.368867 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a6223-8cf7-4758-a4e7-2f225cc22c3f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:44.412194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74ca09-032a-4738-8d74-08d05e792c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:44.413340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74ca09-032a-4738-8d74-08d05e792c77 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:44.413823 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74ca09-032a-4738-8d74-08d05e792c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:44.414605 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74ca09-032a-4738-8d74-08d05e792c77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:44.418006 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:44.420690 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a59138-161b-4645-94ec-f131f0bf10b8 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:44.430301 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74ca09-032a-4738-8d74-08d05e792c77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:44.432232 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74ca09-032a-4738-8d74-08d05e792c77 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:44.449714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-0b74ca09-032a-4738-8d74-08d05e792c77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:44.449714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:44.449714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:44.449714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:44.449714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:44.449714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:44.449714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:44.449714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:44.449714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:44.449714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:44.449714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:44.449714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:44.449714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:51:44.449714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:51:44.449714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:51:44.449714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:51:44.450642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:44.450642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:44.450642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:44.450642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:44.450642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:44.450642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:44.450642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:44.450642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:44.450642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:44.450642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:44.450642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:44.450642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:44.450642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:44.450642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:44.450642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:44.450642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:44.450642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:51:44.451770 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:51:44.451770 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:44.451770 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:44.451770 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 235/473] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:44.457827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74ca09-032a-4738-8d74-08d05e792c77 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:44.462247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:44.481329 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5c0cd-ba98-47ad-a09d-c3f7c2842bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:44.482140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5c0cd-ba98-47ad-a09d-c3f7c2842bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:44.483639 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5c0cd-ba98-47ad-a09d-c3f7c2842bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:44.484424 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5c0cd-ba98-47ad-a09d-c3f7c2842bf8 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:44.486848 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5c0cd-ba98-47ad-a09d-c3f7c2842bf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:44.487328 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5c0cd-ba98-47ad-a09d-c3f7c2842bf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:44.490560 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5c0cd-ba98-47ad-a09d-c3f7c2842bf8 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:44.503604 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 239/474] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:44.504610 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5c0cd-ba98-47ad-a09d-c3f7c2842bf8 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:44.593321 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74ca09-032a-4738-8d74-08d05e792c77 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:44.630333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49629acf-be80-4db0-acab-7e7cecf74123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:44.631053 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49629acf-be80-4db0-acab-7e7cecf74123 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:44.631413 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49629acf-be80-4db0-acab-7e7cecf74123 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:44.631824 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49629acf-be80-4db0-acab-7e7cecf74123 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:44.632650 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49629acf-be80-4db0-acab-7e7cecf74123 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:44.633052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49629acf-be80-4db0-acab-7e7cecf74123 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:44.634685 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49629acf-be80-4db0-acab-7e7cecf74123 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:44.634817 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5c0cd-ba98-47ad-a09d-c3f7c2842bf8 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:44.650539 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 236/475] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:44.651372 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49629acf-be80-4db0-acab-7e7cecf74123 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:44.675621 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a19607-22cd-4bfc-a378-8e472398f501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:44.676398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a19607-22cd-4bfc-a378-8e472398f501 None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:44.676781 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a19607-22cd-4bfc-a378-8e472398f501 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:44.677163 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a19607-22cd-4bfc-a378-8e472398f501 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:44.677938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a19607-22cd-4bfc-a378-8e472398f501 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:44.679774 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a19607-22cd-4bfc-a378-8e472398f501 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:44.691660 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.sql.core [None req-59a19607-22cd-4bfc-a378-8e472398f501 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 14 00:51:44.691660 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 14 00:51:44.691660 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [parameters: {'id': 'c48a9f166015463abcc877e56f77bf3a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 14 00:51:44.691660 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81418) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 14 00:51:44.714192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-59a19607-22cd-4bfc-a378-8e472398f501 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 14 00:51:44.714192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:44.714192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 00:51:44.714192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 00:51:44.714192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 00:51:44.714192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 00:51:44.714192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 00:51:44.714192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 00:51:44.714192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 00:51:44.714192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application conn.query(q) Sep 14 00:51:44.714192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 00:51:44.714192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 00:51:44.714192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 00:51:44.714192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application result.read() Sep 14 00:51:44.714192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 00:51:44.715530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 00:51:44.715530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 00:51:44.715530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 00:51:44.715530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 00:51:44.715530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 00:51:44.715530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 00:51:44.715530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 00:51:44.715530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 14 00:51:44.715530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:44.715530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 14 00:51:44.715530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:44.715530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:44.715530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 14 00:51:44.715530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 14 00:51:44.715530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 14 00:51:44.715530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return ref.to_dict() Sep 14 00:51:44.715530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 00:51:44.715530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application next(self.gen) Sep 14 00:51:44.716468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 14 00:51:44.716468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application yield resource Sep 14 00:51:44.716468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 00:51:44.716468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application next(self.gen) Sep 14 00:51:44.716468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 14 00:51:44.716468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self.session.rollback() Sep 14 00:51:44.716468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 14 00:51:44.716468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self.force_reraise() Sep 14 00:51:44.716468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 14 00:51:44.716468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise self.value Sep 14 00:51:44.716468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 14 00:51:44.716468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 14 00:51:44.716468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 14 00:51:44.716468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application session.commit() Sep 14 00:51:44.716468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 14 00:51:44.716468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 14 00:51:44.716468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 14 00:51:44.717352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._prepare_impl() Sep 14 00:51:44.717352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 14 00:51:44.717352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self.session.flush() Sep 14 00:51:44.717352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 14 00:51:44.717352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._flush(objects) Sep 14 00:51:44.717352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 14 00:51:44.717352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 14 00:51:44.717352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 14 00:51:44.717352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application compat.raise_( Sep 14 00:51:44.717352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 00:51:44.717352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception Sep 14 00:51:44.717352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 14 00:51:44.717352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application flush_context.execute() Sep 14 00:51:44.717352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 14 00:51:44.717352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rec.execute(self) Sep 14 00:51:44.717352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 14 00:51:44.717352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 14 00:51:44.718211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 14 00:51:44.718211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 14 00:51:44.718211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 14 00:51:44.718211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 14 00:51:44.718211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 14 00:51:44.718211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 14 00:51:44.718211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 14 00:51:44.718211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 14 00:51:44.718211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 14 00:51:44.718211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 14 00:51:44.718211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 14 00:51:44.718211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 14 00:51:44.718211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 14 00:51:44.718211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 14 00:51:44.718211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 00:51:44.718211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception Sep 14 00:51:44.719051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 00:51:44.719051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 00:51:44.719051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 00:51:44.719051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 00:51:44.719051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 00:51:44.719051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 00:51:44.719051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 00:51:44.719051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application conn.query(q) Sep 14 00:51:44.719051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 00:51:44.719051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 00:51:44.719051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 00:51:44.719051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application result.read() Sep 14 00:51:44.719051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 00:51:44.719051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 00:51:44.719051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 00:51:44.719051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 00:51:44.721333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 00:51:44.721333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 00:51:44.721333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 00:51:44.721333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 00:51:44.721333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 14 00:51:44.721333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 14 00:51:44.721333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [parameters: {'id': 'c48a9f166015463abcc877e56f77bf3a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 14 00:51:44.721333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 14 00:51:44.721333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:44.721333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:51:44.721333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:44.721333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:44.721333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:44.721333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:44.721333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:44.721333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:44.722655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:44.722655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:44.722655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:44.722655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:44.722655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:44.722655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:44.722655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 14 00:51:44.722655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 14 00:51:44.722655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:44.722655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:44.722655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 14 00:51:44.722655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 14 00:51:44.722655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 14 00:51:44.722655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 14 00:51:44.722655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 14 00:51:44.722655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:44.735899 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 240/476] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Sep 14 00:51:44 2022] POST /identity/v3/roles => generated 144 bytes in 212 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:44.735899 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a19607-22cd-4bfc-a378-8e472398f501 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:44.736080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:44.759137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:44.789056 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49629acf-be80-4db0-acab-7e7cecf74123 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:44.831789 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339173f-6b5f-437d-b191-fdeddfacc63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:44.832703 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339173f-6b5f-437d-b191-fdeddfacc63e None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:44.833162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339173f-6b5f-437d-b191-fdeddfacc63e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:44.833640 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339173f-6b5f-437d-b191-fdeddfacc63e None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:44.834559 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b339173f-6b5f-437d-b191-fdeddfacc63e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:44.836274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b339173f-6b5f-437d-b191-fdeddfacc63e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:44.847909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a19607-22cd-4bfc-a378-8e472398f501 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:44.864122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 237/477] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 00:51:44 2022] POST /identity/v3/users => generated 248 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:51:44.865316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339173f-6b5f-437d-b191-fdeddfacc63e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:44.885817 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220a0a4-9109-42d3-8e2c-d5600c10fe43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:44.886841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220a0a4-9109-42d3-8e2c-d5600c10fe43 None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:44.886986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220a0a4-9109-42d3-8e2c-d5600c10fe43 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:44.887374 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220a0a4-9109-42d3-8e2c-d5600c10fe43 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:44.888245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2220a0a4-9109-42d3-8e2c-d5600c10fe43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:44.890063 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2220a0a4-9109-42d3-8e2c-d5600c10fe43 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:44.930671 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 241/478] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 00:51:44 2022] POST /identity/v3/users => generated 249 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:51:44.995968 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339173f-6b5f-437d-b191-fdeddfacc63e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:45.030604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:45.056844 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851358b-6587-4f64-b758-d5882c9a7bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:45.057504 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851358b-6587-4f64-b758-d5882c9a7bfa None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:45.057785 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851358b-6587-4f64-b758-d5882c9a7bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:45.058062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851358b-6587-4f64-b758-d5882c9a7bfa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:45.076044 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851358b-6587-4f64-b758-d5882c9a7bfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:45.083597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-d851358b-6587-4f64-b758-d5882c9a7bfa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:51:45.083597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:45.083597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:45.083597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:45.083597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:45.083597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:45.083597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:45.083597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:45.083597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:45.083597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:45.083597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:45.083597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:45.083597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:45.083597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:45.083597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:45.083597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:45.084558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:45.084558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:45.084558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:45.084558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:45.084558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:45.084558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:45.084558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:45.084558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:45.084558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:45.084558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:45.084558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:45.084558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:45.084558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:45.084558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:45.084558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:45.084558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:45.085403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:45.085403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:45.085403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:45.085403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:45.085403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:45.085403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:45.085403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:45.085403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:45.085403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:45.085403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:45.085403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 00:51:45.085403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:45.085403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 238/479] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 00:51:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:45.091718 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220a0a4-9109-42d3-8e2c-d5600c10fe43 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:45.203806 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:45.231047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220a0a4-9109-42d3-8e2c-d5600c10fe43 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:45.269060 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0e2ac-aa0e-4741-afc0-993f3fe4b41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:45.269692 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0e2ac-aa0e-4741-afc0-993f3fe4b41a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:45.269975 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0e2ac-aa0e-4741-afc0-993f3fe4b41a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:45.270252 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0e2ac-aa0e-4741-afc0-993f3fe4b41a None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:45.270930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba0e2ac-aa0e-4741-afc0-993f3fe4b41a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:45.271234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba0e2ac-aa0e-4741-afc0-993f3fe4b41a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:45.272717 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba0e2ac-aa0e-4741-afc0-993f3fe4b41a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:45.284001 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 242/480] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:51:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:46.309521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1493e-5c72-4ffb-9dfb-289286d0f2ee None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:46.310077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1493e-5c72-4ffb-9dfb-289286d0f2ee None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:46.310077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1493e-5c72-4ffb-9dfb-289286d0f2ee None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:46.311662 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 239/481] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:46.317496 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63338cc-52fa-4815-a1aa-19c980db3246 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:46.317910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63338cc-52fa-4815-a1aa-19c980db3246 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:46.318101 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63338cc-52fa-4815-a1aa-19c980db3246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:46.337920 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-b63338cc-52fa-4815-a1aa-19c980db3246 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:46.339114 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b63338cc-52fa-4815-a1aa-19c980db3246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:46.410701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 243/482] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:51:46.769885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1493e-5c72-4ffb-9dfb-289286d0f2ee None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:46.899028 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1493e-5c72-4ffb-9dfb-289286d0f2ee None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:46.935465 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038af1bf-c552-4f68-8dbd-648f86837c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:46.936120 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038af1bf-c552-4f68-8dbd-648f86837c63 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:46.936399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038af1bf-c552-4f68-8dbd-648f86837c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:46.936666 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038af1bf-c552-4f68-8dbd-648f86837c63 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:46.949335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038af1bf-c552-4f68-8dbd-648f86837c63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:46.954333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-038af1bf-c552-4f68-8dbd-648f86837c63 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:51:46.954333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:46.954333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:46.954333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:46.954333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:46.954333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:46.954333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:46.954333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:46.954333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:46.954333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:46.954333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:46.954333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:46.954333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:46.954333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:46.954333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:46.954333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:46.955289 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:46.955289 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:46.955289 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:46.955289 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:46.955289 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:46.955289 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:46.955289 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:46.955289 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:46.955289 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:46.955289 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:46.955289 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:46.955289 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:46.955289 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:46.955289 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:46.955289 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:46.955289 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:46.956114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:46.956114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:46.956114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:46.956114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:46.956114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:46.956114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:46.956114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:46.956114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:46.956114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:46.956114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:46.956114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:51:46.956114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:46.956729 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 240/483] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 00:51:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:46.962059 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63338cc-52fa-4815-a1aa-19c980db3246 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:47.069451 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:47.101931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8f2ff-1b3c-46b6-9046-afb06dfc685a None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:47.102265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8f2ff-1b3c-46b6-9046-afb06dfc685a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:47.102491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8f2ff-1b3c-46b6-9046-afb06dfc685a None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:47.103988 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 241/484] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:51:47.112353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b39591-a94b-443f-aa4e-3ca176f136cf None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:47.112883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b39591-a94b-443f-aa4e-3ca176f136cf None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:47.113374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b39591-a94b-443f-aa4e-3ca176f136cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:47.114213 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63338cc-52fa-4815-a1aa-19c980db3246 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:47.133922 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e4b39591-a94b-443f-aa4e-3ca176f136cf None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:47.136154 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e4b39591-a94b-443f-aa4e-3ca176f136cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:47.182051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e462b-9567-41b0-b150-4b9f8ff77145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:47.183032 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e462b-9567-41b0-b150-4b9f8ff77145 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:47.183519 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e462b-9567-41b0-b150-4b9f8ff77145 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:47.184010 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e462b-9567-41b0-b150-4b9f8ff77145 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:47.184894 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e462b-9567-41b0-b150-4b9f8ff77145 None admin] RBAC: Adding query filter params (name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:47.185394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e462b-9567-41b0-b150-4b9f8ff77145 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:47.187644 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e462b-9567-41b0-b150-4b9f8ff77145 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:47.205080 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 244/485] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 00:51:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:47.209157 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:47.214702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e462b-9567-41b0-b150-4b9f8ff77145 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:47.267579 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 242/486] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:51:47.365267 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e462b-9567-41b0-b150-4b9f8ff77145 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:47.395463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:47.406051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7167aa2-2b25-4152-b966-2b219a3e0044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:47.406759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7167aa2-2b25-4152-b966-2b219a3e0044 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:47.407066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7167aa2-2b25-4152-b966-2b219a3e0044 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:47.407577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7167aa2-2b25-4152-b966-2b219a3e0044 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:47.423921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7167aa2-2b25-4152-b966-2b219a3e0044 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:47.429108 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7167aa2-2b25-4152-b966-2b219a3e0044 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:47.444420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-f7167aa2-2b25-4152-b966-2b219a3e0044 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:47.444420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:47.444420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:47.444420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:47.444420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:47.444420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:47.444420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:47.444420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:47.444420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:47.444420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:47.444420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:47.444420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:47.444420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:51:47.444420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:51:47.444420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:51:47.444420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:51:47.445345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:47.445345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:47.445345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:47.445345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:47.445345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:47.445345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:47.445345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:47.445345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:47.445345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:47.445345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:47.445345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:47.445345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:47.445345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:47.445345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:47.445345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:47.445345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:47.445345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:51:47.446193 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:51:47.446193 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:47.446193 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:47.446193 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 245/487] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:47.451138 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b39591-a94b-443f-aa4e-3ca176f136cf None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:47.580059 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b39591-a94b-443f-aa4e-3ca176f136cf None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:47.623283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0fb15-45e3-471a-b7fa-b4c290b10b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:47.623854 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0fb15-45e3-471a-b7fa-b4c290b10b6a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:47.624135 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0fb15-45e3-471a-b7fa-b4c290b10b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:47.624406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0fb15-45e3-471a-b7fa-b4c290b10b6a None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:47.625061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0fb15-45e3-471a-b7fa-b4c290b10b6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:47.625365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0fb15-45e3-471a-b7fa-b4c290b10b6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:47.627018 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0fb15-45e3-471a-b7fa-b4c290b10b6a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:47.639606 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 243/488] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:47.648621 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7167aa2-2b25-4152-b966-2b219a3e0044 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:47.690519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0fb15-45e3-471a-b7fa-b4c290b10b6a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:47.733726 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:47.773710 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7167aa2-2b25-4152-b966-2b219a3e0044 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:47.808780 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b413c-4bf0-4b91-90e5-11e821f3804a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:47.809415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b413c-4bf0-4b91-90e5-11e821f3804a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:47.809695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b413c-4bf0-4b91-90e5-11e821f3804a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:47.809969 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b413c-4bf0-4b91-90e5-11e821f3804a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:47.817697 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14b413c-4bf0-4b91-90e5-11e821f3804a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:47.819153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14b413c-4bf0-4b91-90e5-11e821f3804a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:47.822033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0fb15-45e3-471a-b7fa-b4c290b10b6a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:47.832150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-b14b413c-4bf0-4b91-90e5-11e821f3804a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 00:51:47.832150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:47.832150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:47.832150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:47.832150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:47.832150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:47.832150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:47.832150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:47.832150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:47.832150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:47.832150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:47.832150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:47.832150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:51:47.832150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:51:47.832150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:51:47.832150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:51:47.833066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:47.833066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:47.833066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:51:47.833066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:51:47.833066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:51:47.833066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:51:47.833066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:47.833066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:47.833066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:47.833066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:47.833066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:47.833066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:47.833066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:47.833066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:47.833066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:47.833066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:47.833066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:47.833066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:47.833968 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:47.833968 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:47.833968 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:51:47.833968 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:51:47.833968 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:51:47.833968 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:51:47.833968 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:51:47.833968 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:51:47.833968 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 00:51:47.833968 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:47.833968 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 246/489] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Sep 14 00:51:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:47.835026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348dbc06-1c9f-4b7a-a865-c4cfeb0f8e82 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:47.835026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348dbc06-1c9f-4b7a-a865-c4cfeb0f8e82 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:47.835246 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348dbc06-1c9f-4b7a-a865-c4cfeb0f8e82 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:47.839598 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 247/490] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:47.839598 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f1d89-fe3e-4d29-8cef-3f4b4907ae21 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:47.839598 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f1d89-fe3e-4d29-8cef-3f4b4907ae21 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:47.839598 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f1d89-fe3e-4d29-8cef-3f4b4907ae21 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:47.839598 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 248/491] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:47.844154 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f1d89-fe3e-4d29-8cef-3f4b4907ae21 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:47.864852 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d907657-5dc5-4b4c-bda9-d24a66c080db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:47.865486 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d907657-5dc5-4b4c-bda9-d24a66c080db None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:47.865767 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d907657-5dc5-4b4c-bda9-d24a66c080db None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:47.866038 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d907657-5dc5-4b4c-bda9-d24a66c080db None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:47.877676 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d907657-5dc5-4b4c-bda9-d24a66c080db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:47.882827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-3d907657-5dc5-4b4c-bda9-d24a66c080db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:51:47.882827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:47.882827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:47.882827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:47.882827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:47.882827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:47.882827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:47.882827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:47.882827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:47.882827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:47.882827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:47.882827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:47.882827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:47.882827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:47.882827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:47.882827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:47.883788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:47.883788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:47.883788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:47.883788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:47.883788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:47.883788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:47.883788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:47.883788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:47.883788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:47.883788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:47.883788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:47.883788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:47.883788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:47.883788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:47.883788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:47.883788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:47.884600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:47.884600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:47.884600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:47.884600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:47.884600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:47.884600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:47.884600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:47.884600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:47.884600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:47.884600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:47.884600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:51:47.884600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:47.884600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 244/492] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 00:51:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:51:47.885926 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3a766-e982-49bd-a979-0fc213c5ce56 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:47.886027 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3a766-e982-49bd-a979-0fc213c5ce56 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:47.886322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3a766-e982-49bd-a979-0fc213c5ce56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:47.897738 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:47.905692 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e5b3a766-e982-49bd-a979-0fc213c5ce56 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:47.906900 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e5b3a766-e982-49bd-a979-0fc213c5ce56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:47.958774 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f1d89-fe3e-4d29-8cef-3f4b4907ae21 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:47.991778 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 245/493] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:47.993851 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db0e4c-746d-43fa-bc8a-abed2a3a64db None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:47.994125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34df299-08b7-4887-a095-4d43bd30d626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:47.994600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db0e4c-746d-43fa-bc8a-abed2a3a64db None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:47.994773 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34df299-08b7-4887-a095-4d43bd30d626 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:47.995015 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34df299-08b7-4887-a095-4d43bd30d626 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:47.995289 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34df299-08b7-4887-a095-4d43bd30d626 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:47.995667 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db0e4c-746d-43fa-bc8a-abed2a3a64db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:47.996341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34df299-08b7-4887-a095-4d43bd30d626 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:47.996876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34df299-08b7-4887-a095-4d43bd30d626 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:47.998375 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34df299-08b7-4887-a095-4d43bd30d626 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:48.020646 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f4db0e4c-746d-43fa-bc8a-abed2a3a64db None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:48.022162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f4db0e4c-746d-43fa-bc8a-abed2a3a64db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:48.040716 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:48.042577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 249/494] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Sep 14 00:51:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:48.043532 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34df299-08b7-4887-a095-4d43bd30d626 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:48.129607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 246/495] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:48.129607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db0e4c-746d-43fa-bc8a-abed2a3a64db None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:48.171014 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:48.173839 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34df299-08b7-4887-a095-4d43bd30d626 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:48.241217 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a466c4d-45e4-45f0-969c-6337a63c88a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:48.241904 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a466c4d-45e4-45f0-969c-6337a63c88a6 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:48.242188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a466c4d-45e4-45f0-969c-6337a63c88a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:48.242505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a466c4d-45e4-45f0-969c-6337a63c88a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:48.243180 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a466c4d-45e4-45f0-969c-6337a63c88a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:48.243485 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a466c4d-45e4-45f0-969c-6337a63c88a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:48.244910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a466c4d-45e4-45f0-969c-6337a63c88a6 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:48.264039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 250/496] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 00:51:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:48.264883 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a466c4d-45e4-45f0-969c-6337a63c88a6 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:48.292017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db0e4c-746d-43fa-bc8a-abed2a3a64db None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:48.331251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60e37b-b2d2-4e3b-887f-db7c7a89bfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:48.332031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60e37b-b2d2-4e3b-887f-db7c7a89bfd4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:48.332314 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60e37b-b2d2-4e3b-887f-db7c7a89bfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:48.332703 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60e37b-b2d2-4e3b-887f-db7c7a89bfd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:48.354283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:48.385117 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a466c4d-45e4-45f0-969c-6337a63c88a6 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:48.415852 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa60e37b-b2d2-4e3b-887f-db7c7a89bfd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:48.417614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa60e37b-b2d2-4e3b-887f-db7c7a89bfd4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:48.422088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058944d-2f5a-47f8-9021-987b8ead953c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:48.422778 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058944d-2f5a-47f8-9021-987b8ead953c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:48.423068 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058944d-2f5a-47f8-9021-987b8ead953c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:48.423349 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058944d-2f5a-47f8-9021-987b8ead953c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:48.434647 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058944d-2f5a-47f8-9021-987b8ead953c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:48.442698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-a058944d-2f5a-47f8-9021-987b8ead953c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:51:48.442698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:48.442698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:48.442698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:48.442698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:48.442698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:48.442698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:48.442698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:48.442698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:48.442698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:48.442698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:48.442698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:48.442698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:48.442698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:48.442698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:48.442698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:48.443695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:48.443695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:48.443695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:48.443695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:48.443695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:48.443695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:48.443695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:48.443695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:48.443695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:48.443695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:48.443695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:48.443695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:48.443695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:48.443695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:48.443695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:48.443695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:48.444717 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:48.444717 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:48.444717 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:48.444717 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:48.444717 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:48.444717 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:48.444717 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:48.444717 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:48.444717 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:48.444717 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:48.444717 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:51:48.444717 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:48.444717 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 251/497] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 00:51:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:48.444717 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058944d-2f5a-47f8-9021-987b8ead953c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:48.469427 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:48.527029 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 247/498] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:51:48.527907 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60e37b-b2d2-4e3b-887f-db7c7a89bfd4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:48.548422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058944d-2f5a-47f8-9021-987b8ead953c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:48.577620 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:48.581580 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bdc096-ea34-4e1f-af62-6ac1a8eec3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:48.582395 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdc096-ea34-4e1f-af62-6ac1a8eec3b1 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:48.582674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdc096-ea34-4e1f-af62-6ac1a8eec3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:48.582954 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdc096-ea34-4e1f-af62-6ac1a8eec3b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:48.593428 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bdc096-ea34-4e1f-af62-6ac1a8eec3b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:48.595730 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bdc096-ea34-4e1f-af62-6ac1a8eec3b1 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:48.609986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-a2bdc096-ea34-4e1f-af62-6ac1a8eec3b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:48.609986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:48.609986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:48.609986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:48.609986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:48.609986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:48.609986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:48.609986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:48.609986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:48.609986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:48.609986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:48.609986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:48.609986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:51:48.609986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:51:48.609986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:51:48.609986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:51:48.610967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:48.610967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:48.610967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:48.610967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:48.610967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:48.610967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:48.610967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:48.610967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:48.610967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:48.610967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:48.610967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:48.610967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:48.610967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:48.610967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:48.610967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:48.610967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:48.610967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:51:48.611851 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:51:48.611851 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:48.611851 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:48.611851 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 252/499] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:48.612763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad836e9-8863-41fc-96e0-a47095375f28 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:48.613046 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad836e9-8863-41fc-96e0-a47095375f28 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:48.613353 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad836e9-8863-41fc-96e0-a47095375f28 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:48.614574 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 253/500] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:48.615272 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad836e9-8863-41fc-96e0-a47095375f28 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:48.633254 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60e37b-b2d2-4e3b-887f-db7c7a89bfd4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:48.667401 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b177bfe-7e25-43a6-bc82-a9cc5d688127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:48.668050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b177bfe-7e25-43a6-bc82-a9cc5d688127 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:48.668329 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b177bfe-7e25-43a6-bc82-a9cc5d688127 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:48.668601 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b177bfe-7e25-43a6-bc82-a9cc5d688127 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:48.678757 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b177bfe-7e25-43a6-bc82-a9cc5d688127 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:48.683668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-6b177bfe-7e25-43a6-bc82-a9cc5d688127 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:51:48.683668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:48.683668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:48.683668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:48.683668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:48.683668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:48.683668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:48.683668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:48.683668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:48.683668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:48.683668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:48.683668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:48.683668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:48.683668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:48.683668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:48.683668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:48.684788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:48.684788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:48.684788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:48.684788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:48.684788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:48.684788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:48.684788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:48.684788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:48.684788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:48.684788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:48.684788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:48.684788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:48.684788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:48.684788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:48.684788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:48.684788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:48.685876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:48.685876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:48.685876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:48.685876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:48.685876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:48.685876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:48.685876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:48.685876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:48.685876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:48.685876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:48.685876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:51:48.685876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:48.685876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 248/501] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 00:51:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:48.685876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b177bfe-7e25-43a6-bc82-a9cc5d688127 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:48.695843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:48.722120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad836e9-8863-41fc-96e0-a47095375f28 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:48.755833 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e81bf-9835-4f0d-a7ce-3fdc0fed32f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:48.756444 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e81bf-9835-4f0d-a7ce-3fdc0fed32f4 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:48.756727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e81bf-9835-4f0d-a7ce-3fdc0fed32f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:48.757005 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e81bf-9835-4f0d-a7ce-3fdc0fed32f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:48.757664 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e81bf-9835-4f0d-a7ce-3fdc0fed32f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:48.757970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e81bf-9835-4f0d-a7ce-3fdc0fed32f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:48.759375 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e81bf-9835-4f0d-a7ce-3fdc0fed32f4 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:48.768049 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 254/502] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 00:51:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:48.768851 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e81bf-9835-4f0d-a7ce-3fdc0fed32f4 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:48.786343 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b177bfe-7e25-43a6-bc82-a9cc5d688127 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:48.796980 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:48.823434 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5460fa7e-f9ca-44a4-baf6-e10e7487e138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:48.824333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460fa7e-f9ca-44a4-baf6-e10e7487e138 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:48.824712 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460fa7e-f9ca-44a4-baf6-e10e7487e138 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:48.825084 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460fa7e-f9ca-44a4-baf6-e10e7487e138 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:48.836112 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5460fa7e-f9ca-44a4-baf6-e10e7487e138 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:48.837569 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5460fa7e-f9ca-44a4-baf6-e10e7487e138 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:48.853123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-5460fa7e-f9ca-44a4-baf6-e10e7487e138 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:48.853123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:48.853123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:48.853123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:48.853123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:48.853123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:48.853123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:48.853123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:48.853123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:48.853123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:48.853123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:48.853123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:48.853123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:51:48.853123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:51:48.853123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:51:48.853123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:51:48.854052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:48.854052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:48.854052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:48.854052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:48.854052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:48.854052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:48.854052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:48.854052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:48.854052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:48.854052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:48.854052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:48.854052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:48.854052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:48.854052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:48.854052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:48.854052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:48.854052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:51:48.854899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:51:48.854899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:48.854899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:48.854899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 249/503] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:48.855390 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3a34d-9480-402b-a1d6-765616ec753f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:48.855608 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3a34d-9480-402b-a1d6-765616ec753f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:48.855882 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3a34d-9480-402b-a1d6-765616ec753f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:48.870754 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e81bf-9835-4f0d-a7ce-3fdc0fed32f4 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:48.873140 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-70a3a34d-9480-402b-a1d6-765616ec753f None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:48.874394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-70a3a34d-9480-402b-a1d6-765616ec753f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:48.904729 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04cbe7-c5b2-4298-89ca-c5b017ae1237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:48.905339 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04cbe7-c5b2-4298-89ca-c5b017ae1237 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:48.905621 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04cbe7-c5b2-4298-89ca-c5b017ae1237 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:48.905900 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04cbe7-c5b2-4298-89ca-c5b017ae1237 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:48.906537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04cbe7-c5b2-4298-89ca-c5b017ae1237 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:48.906851 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04cbe7-c5b2-4298-89ca-c5b017ae1237 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:48.908862 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04cbe7-c5b2-4298-89ca-c5b017ae1237 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:48.911495 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:48.921310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 255/504] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:51:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:48.922052 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04cbe7-c5b2-4298-89ca-c5b017ae1237 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:48.942643 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 250/505] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:48.943354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a3a34d-9480-402b-a1d6-765616ec753f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:49.022014 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04cbe7-c5b2-4298-89ca-c5b017ae1237 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:49.056129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:49.059828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bbf373-4b5e-42b4-99c8-9e5826a06c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:49.060449 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbf373-4b5e-42b4-99c8-9e5826a06c8f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:49.060732 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbf373-4b5e-42b4-99c8-9e5826a06c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:49.061008 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbf373-4b5e-42b4-99c8-9e5826a06c8f None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:49.061675 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bbf373-4b5e-42b4-99c8-9e5826a06c8f None admin] RBAC: Adding query filter params (name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:49.061986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bbf373-4b5e-42b4-99c8-9e5826a06c8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:49.063438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bbf373-4b5e-42b4-99c8-9e5826a06c8f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:49.065883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a3a34d-9480-402b-a1d6-765616ec753f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:49.071849 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 256/506] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 00:51:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:49.072649 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bbf373-4b5e-42b4-99c8-9e5826a06c8f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:49.107535 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952301c-b7e3-4215-ba47-ab2982609ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:49.108190 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952301c-b7e3-4215-ba47-ab2982609ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:49.108610 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952301c-b7e3-4215-ba47-ab2982609ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:49.108893 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952301c-b7e3-4215-ba47-ab2982609ed2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:49.120102 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2952301c-b7e3-4215-ba47-ab2982609ed2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:49.121791 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2952301c-b7e3-4215-ba47-ab2982609ed2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:49.132752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:49.139813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-2952301c-b7e3-4215-ba47-ab2982609ed2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:49.139813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:49.139813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:49.139813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:49.139813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:49.139813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:49.139813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:49.139813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:49.139813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:49.139813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:49.139813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:49.139813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:49.139813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:51:49.139813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:51:49.139813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:51:49.139813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:51:49.140978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:49.140978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:49.140978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:49.140978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:49.140978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:49.140978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:49.140978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:49.140978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:49.140978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:49.140978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:49.140978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:49.140978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:49.140978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:49.140978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:49.140978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:49.140978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:49.140978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:51:49.142041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:51:49.142041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:49.142041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:49.142041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 251/507] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:49.142401 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952301c-b7e3-4215-ba47-ab2982609ed2 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:49.180641 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bbf373-4b5e-42b4-99c8-9e5826a06c8f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:49.214775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e992fc5-4483-4e10-baec-41e94fc74d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:49.215390 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e992fc5-4483-4e10-baec-41e94fc74d4a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:49.215721 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e992fc5-4483-4e10-baec-41e94fc74d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:49.215993 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e992fc5-4483-4e10-baec-41e94fc74d4a None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:49.216632 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e992fc5-4483-4e10-baec-41e94fc74d4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:49.216938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e992fc5-4483-4e10-baec-41e94fc74d4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:49.218315 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e992fc5-4483-4e10-baec-41e94fc74d4a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:49.230908 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 257/508] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:49.231845 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e992fc5-4483-4e10-baec-41e94fc74d4a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:49.258044 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952301c-b7e3-4215-ba47-ab2982609ed2 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:49.291378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c088890-7712-4c40-9c5a-ede5a06a53af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:49.292044 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c088890-7712-4c40-9c5a-ede5a06a53af None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:49.292336 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c088890-7712-4c40-9c5a-ede5a06a53af None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:49.292617 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c088890-7712-4c40-9c5a-ede5a06a53af None admin] PATH_INFO: `/v3/users/glance` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:49.300593 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c088890-7712-4c40-9c5a-ede5a06a53af None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:49.302062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c088890-7712-4c40-9c5a-ede5a06a53af None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:49.314334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-9c088890-7712-4c40-9c5a-ede5a06a53af None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 14 00:51:49.314334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:49.314334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:49.314334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:49.314334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:49.314334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:49.314334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:49.314334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:49.314334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:49.314334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:49.314334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:49.314334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:49.314334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:51:49.314334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:51:49.314334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:51:49.314334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:51:49.315354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:49.315354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:49.315354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:51:49.315354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:51:49.315354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:51:49.315354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:51:49.315354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:49.315354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:49.315354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:49.315354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:49.315354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:49.315354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:49.315354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:49.315354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:49.315354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:49.315354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:49.315354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:49.315354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:49.316873 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:49.316873 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:49.316873 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:51:49.316873 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:51:49.316873 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:51:49.316873 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:51:49.316873 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:51:49.316873 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:51:49.316873 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 14 00:51:49.316873 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:49.316873 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 252/509] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:49.316873 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c088890-7712-4c40-9c5a-ede5a06a53af None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:49.319490 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:49.335724 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e992fc5-4483-4e10-baec-41e94fc74d4a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:49.353371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:49.370184 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0802671-94e6-4578-978e-e4e6ad4ac409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:49.370812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0802671-94e6-4578-978e-e4e6ad4ac409 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:49.371090 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0802671-94e6-4578-978e-e4e6ad4ac409 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:49.371363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0802671-94e6-4578-978e-e4e6ad4ac409 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:49.381474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0802671-94e6-4578-978e-e4e6ad4ac409 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:49.382930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0802671-94e6-4578-978e-e4e6ad4ac409 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:49.398161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-b0802671-94e6-4578-978e-e4e6ad4ac409 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:49.398161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:49.398161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:49.398161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:49.398161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:49.398161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:49.398161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:49.398161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:49.398161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:49.398161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:49.398161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:49.398161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:49.398161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:51:49.398161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:51:49.398161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:51:49.398161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:51:49.399474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:49.399474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:49.399474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:49.399474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:49.399474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:49.399474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:49.399474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:49.399474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:49.399474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:49.399474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:49.399474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:49.399474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:49.399474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:49.399474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:49.399474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:49.399474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:49.399474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:51:49.400389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:51:49.400389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:49.400389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:49.400389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 258/510] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:49.400389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0802671-94e6-4578-978e-e4e6ad4ac409 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:49.420473 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c088890-7712-4c40-9c5a-ede5a06a53af None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:49.454636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4eb7cd-9337-4a0d-9236-76ae6bdc76f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:49.455179 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4eb7cd-9337-4a0d-9236-76ae6bdc76f7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:49.455461 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4eb7cd-9337-4a0d-9236-76ae6bdc76f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:49.455767 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4eb7cd-9337-4a0d-9236-76ae6bdc76f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:49.456408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4eb7cd-9337-4a0d-9236-76ae6bdc76f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:49.456714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4eb7cd-9337-4a0d-9236-76ae6bdc76f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:49.458102 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4eb7cd-9337-4a0d-9236-76ae6bdc76f7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:49.470524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 253/511] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:49.471354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4eb7cd-9337-4a0d-9236-76ae6bdc76f7 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:49.506097 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0802671-94e6-4578-978e-e4e6ad4ac409 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:49.533022 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:49.541013 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c89798-1b1f-47de-a923-2ce0b047f815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:49.541651 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c89798-1b1f-47de-a923-2ce0b047f815 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:49.541927 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c89798-1b1f-47de-a923-2ce0b047f815 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:49.542197 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c89798-1b1f-47de-a923-2ce0b047f815 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:49.552564 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c89798-1b1f-47de-a923-2ce0b047f815 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:49.554015 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c89798-1b1f-47de-a923-2ce0b047f815 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:49.567930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-89c89798-1b1f-47de-a923-2ce0b047f815 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:49.567930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:49.567930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:49.567930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:49.567930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:49.567930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:49.567930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:49.567930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:49.567930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:49.567930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:49.567930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:49.567930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:49.567930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:51:49.567930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:51:49.567930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:51:49.567930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:51:49.568854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:49.568854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:49.568854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:49.568854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:49.568854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:49.568854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:49.568854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:49.568854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:49.568854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:49.568854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:49.568854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:49.568854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:49.568854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:49.568854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:49.568854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:49.568854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:49.568854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:51:49.569699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:51:49.569699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:49.569699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:49.569699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 259/512] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:49.569910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c89798-1b1f-47de-a923-2ce0b047f815 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:49.576398 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4eb7cd-9337-4a0d-9236-76ae6bdc76f7 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:49.593970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:49.613536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d9fd1-97d3-4831-8f08-8b07397d9dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:49.614167 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d9fd1-97d3-4831-8f08-8b07397d9dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:49.614448 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d9fd1-97d3-4831-8f08-8b07397d9dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:49.614722 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d9fd1-97d3-4831-8f08-8b07397d9dc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:49.626404 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d9fd1-97d3-4831-8f08-8b07397d9dc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:49.628065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d9fd1-97d3-4831-8f08-8b07397d9dc2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:49.640050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-228d9fd1-97d3-4831-8f08-8b07397d9dc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:51:49.640050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:49.640050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:49.640050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:49.640050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:49.640050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:49.640050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:49.640050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:49.640050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:49.640050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:49.640050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:49.640050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:49.640050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:51:49.640050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:51:49.640050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:51:49.641186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:51:49.641186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:49.641186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:49.641186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:49.641186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:49.641186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:49.641186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:49.641186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:49.641186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:49.641186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:49.641186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:49.641186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:49.641186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:49.641186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:49.641186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:49.641186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:49.641186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:49.642602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:51:49.642602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:51:49.642602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:51:49.642602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:51:49.642602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:51:49.642602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:51:49.642602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:51:49.642602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:49.642602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 254/513] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:51:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:49.643110 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d9fd1-97d3-4831-8f08-8b07397d9dc2 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:49.671690 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c89798-1b1f-47de-a923-2ce0b047f815 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:49.706007 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8db6ca-662f-4c8a-9fc4-b756917b786c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:49.706550 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8db6ca-662f-4c8a-9fc4-b756917b786c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:49.706831 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8db6ca-662f-4c8a-9fc4-b756917b786c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:49.707103 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8db6ca-662f-4c8a-9fc4-b756917b786c None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:49.707856 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8db6ca-662f-4c8a-9fc4-b756917b786c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:49.708160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8db6ca-662f-4c8a-9fc4-b756917b786c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:49.709622 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8db6ca-662f-4c8a-9fc4-b756917b786c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:49.727323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 260/514] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Sep 14 00:51:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:49.728168 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8db6ca-662f-4c8a-9fc4-b756917b786c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:49.755631 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d9fd1-97d3-4831-8f08-8b07397d9dc2 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:49.778295 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:49.790029 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b2c34-1f89-4137-8bf4-4ad7ecba177f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:49.790624 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b2c34-1f89-4137-8bf4-4ad7ecba177f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:49.790902 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b2c34-1f89-4137-8bf4-4ad7ecba177f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:49.791168 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b2c34-1f89-4137-8bf4-4ad7ecba177f None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:49.791828 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b2c34-1f89-4137-8bf4-4ad7ecba177f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:49.792126 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b2c34-1f89-4137-8bf4-4ad7ecba177f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:49.793525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b2c34-1f89-4137-8bf4-4ad7ecba177f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:49.806032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 255/515] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:49.810669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:49.810772 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b2c34-1f89-4137-8bf4-4ad7ecba177f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:49.827423 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8db6ca-662f-4c8a-9fc4-b756917b786c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:49.861575 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34ecc6-7b75-49e7-98c5-93e56d912a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:49.862199 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34ecc6-7b75-49e7-98c5-93e56d912a19 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:49.862474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34ecc6-7b75-49e7-98c5-93e56d912a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:49.862748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34ecc6-7b75-49e7-98c5-93e56d912a19 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:49.869514 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34ecc6-7b75-49e7-98c5-93e56d912a19 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:49.870951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34ecc6-7b75-49e7-98c5-93e56d912a19 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:49.882727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-6b34ecc6-7b75-49e7-98c5-93e56d912a19 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 00:51:49.882727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:49.882727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:49.882727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:49.882727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:49.882727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:49.882727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:49.882727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:49.882727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:49.882727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:49.882727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:49.882727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:49.882727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:51:49.882727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:51:49.882727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:51:49.882727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:51:49.883680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:49.883680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:49.883680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:51:49.883680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:51:49.883680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:51:49.883680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:51:49.883680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:49.883680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:49.883680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:49.883680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:49.883680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:49.883680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:49.883680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:49.883680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:49.883680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:49.883680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:49.883680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:49.883680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:49.884585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:49.884585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:49.884585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:51:49.884585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:51:49.884585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:51:49.884585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:51:49.884585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:51:49.884585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:51:49.884585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 00:51:49.884585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:49.884585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 261/516] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 00:51:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:49.884585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34ecc6-7b75-49e7-98c5-93e56d912a19 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:49.913231 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b2c34-1f89-4137-8bf4-4ad7ecba177f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:49.946959 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196e989-4ee1-4e0f-8579-2cf0eabefa45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:49.947585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196e989-4ee1-4e0f-8579-2cf0eabefa45 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:49.947869 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196e989-4ee1-4e0f-8579-2cf0eabefa45 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:49.948144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196e989-4ee1-4e0f-8579-2cf0eabefa45 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:49.948781 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196e989-4ee1-4e0f-8579-2cf0eabefa45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:49.949082 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196e989-4ee1-4e0f-8579-2cf0eabefa45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:49.950465 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196e989-4ee1-4e0f-8579-2cf0eabefa45 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:49.962418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 256/517] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:49.963156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196e989-4ee1-4e0f-8579-2cf0eabefa45 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:49.983310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34ecc6-7b75-49e7-98c5-93e56d912a19 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:50.010458 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:50.017104 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90efce86-e36f-4032-922d-c783f3e81aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:50.017721 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efce86-e36f-4032-922d-c783f3e81aeb None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:50.018010 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efce86-e36f-4032-922d-c783f3e81aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:50.018289 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efce86-e36f-4032-922d-c783f3e81aeb None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:50.018976 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90efce86-e36f-4032-922d-c783f3e81aeb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:50.019287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90efce86-e36f-4032-922d-c783f3e81aeb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:50.020767 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90efce86-e36f-4032-922d-c783f3e81aeb None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:50.035596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:50.039082 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 262/518] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:51:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:50.040361 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90efce86-e36f-4032-922d-c783f3e81aeb None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:50.065075 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196e989-4ee1-4e0f-8579-2cf0eabefa45 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:50.100650 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b92d2-073b-4164-9068-cf6857b1f779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:50.101390 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b92d2-073b-4164-9068-cf6857b1f779 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:50.101680 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b92d2-073b-4164-9068-cf6857b1f779 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:50.101956 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b92d2-073b-4164-9068-cf6857b1f779 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:50.139078 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90efce86-e36f-4032-922d-c783f3e81aeb None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:50.169669 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b92d2-073b-4164-9068-cf6857b1f779 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:50.171118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b92d2-073b-4164-9068-cf6857b1f779 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:50.173094 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382a48f-50c6-4bdc-b910-7297cd51d6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:50.173766 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382a48f-50c6-4bdc-b910-7297cd51d6d1 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:50.174045 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382a48f-50c6-4bdc-b910-7297cd51d6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:50.174323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382a48f-50c6-4bdc-b910-7297cd51d6d1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:50.181343 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382a48f-50c6-4bdc-b910-7297cd51d6d1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:50.182777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382a48f-50c6-4bdc-b910-7297cd51d6d1 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:50.194577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-4382a48f-50c6-4bdc-b910-7297cd51d6d1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 00:51:50.194577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:50.194577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:50.194577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:50.194577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:50.194577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:50.194577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:50.194577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:50.194577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:50.194577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:50.194577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:50.194577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:50.194577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:51:50.194577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:51:50.194577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:51:50.194577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:51:50.195678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:50.195678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:50.195678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:51:50.195678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:51:50.195678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:51:50.195678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:51:50.195678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:50.195678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:50.195678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:50.195678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:50.195678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:50.195678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:50.195678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:50.195678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:50.195678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:50.195678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:50.195678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:50.195678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:50.196781 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:50.196781 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:50.196781 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:51:50.196781 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:51:50.196781 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:51:50.196781 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:51:50.196781 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:51:50.196781 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:51:50.196781 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 00:51:50.196781 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:50.197352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 263/519] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:50.197988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382a48f-50c6-4bdc-b910-7297cd51d6d1 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:50.217570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:50.259524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:50.274143 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 257/520] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:51:50.274960 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b92d2-073b-4164-9068-cf6857b1f779 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:50.300731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382a48f-50c6-4bdc-b910-7297cd51d6d1 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:50.334300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1f861-81e5-4cb5-a21c-da9858af77ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:50.334903 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1f861-81e5-4cb5-a21c-da9858af77ac None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:50.335181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1f861-81e5-4cb5-a21c-da9858af77ac None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:50.335452 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1f861-81e5-4cb5-a21c-da9858af77ac None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:50.336197 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1f861-81e5-4cb5-a21c-da9858af77ac None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:50.336502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1f861-81e5-4cb5-a21c-da9858af77ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:50.337933 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1f861-81e5-4cb5-a21c-da9858af77ac None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:50.355168 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 264/521] 10.222.0.42 () {56 vars in 1139 bytes} [Wed Sep 14 00:51:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:50.355928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1f861-81e5-4cb5-a21c-da9858af77ac None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:50.374214 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b92d2-073b-4164-9068-cf6857b1f779 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:50.410091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece4e19-6def-4c6f-9f46-8c1277474483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:50.410699 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece4e19-6def-4c6f-9f46-8c1277474483 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:50.410981 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece4e19-6def-4c6f-9f46-8c1277474483 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:50.411269 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece4e19-6def-4c6f-9f46-8c1277474483 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:50.411976 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece4e19-6def-4c6f-9f46-8c1277474483 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:50.413429 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece4e19-6def-4c6f-9f46-8c1277474483 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:50.430128 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:50.435099 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 258/522] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 00:51:50 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:51:50.437370 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece4e19-6def-4c6f-9f46-8c1277474483 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:50.466299 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1f861-81e5-4cb5-a21c-da9858af77ac None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:50.490573 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:50.512357 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c24a78-16f4-4323-8b84-7a1c3402747a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:50.513257 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c24a78-16f4-4323-8b84-7a1c3402747a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:50.513543 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c24a78-16f4-4323-8b84-7a1c3402747a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:50.513825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c24a78-16f4-4323-8b84-7a1c3402747a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:50.514575 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c24a78-16f4-4323-8b84-7a1c3402747a None admin] RBAC: Adding query filter params (role.id=7a33c2ceb92a478f81753573f8ece1bd, scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3, user.id=5e0fe2e84f6748c992c9beedf2f5d49b) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:50.514889 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c24a78-16f4-4323-8b84-7a1c3402747a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:50.516690 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c24a78-16f4-4323-8b84-7a1c3402747a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:50.522683 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 265/523] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:51:50 2022] GET /identity/v3/role_assignments?user.id=5e0fe2e84f6748c992c9beedf2f5d49b&scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3&role.id=7a33c2ceb92a478f81753573f8ece1bd => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:50.531823 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c24a78-16f4-4323-8b84-7a1c3402747a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:50.617168 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece4e19-6def-4c6f-9f46-8c1277474483 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:50.652836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c24a78-16f4-4323-8b84-7a1c3402747a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:50.670879 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86320621-02a9-4ba8-89d1-729d93df1327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:50.671514 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86320621-02a9-4ba8-89d1-729d93df1327 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:50.671832 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86320621-02a9-4ba8-89d1-729d93df1327 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:50.672111 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86320621-02a9-4ba8-89d1-729d93df1327 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:50.672935 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86320621-02a9-4ba8-89d1-729d93df1327 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:50.673840 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86320621-02a9-4ba8-89d1-729d93df1327 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:50.674703 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86320621-02a9-4ba8-89d1-729d93df1327 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:50.701644 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892573e-b3f3-4b3c-aad7-53da623065e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:50.702311 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892573e-b3f3-4b3c-aad7-53da623065e1 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:50.702592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892573e-b3f3-4b3c-aad7-53da623065e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:50.702864 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892573e-b3f3-4b3c-aad7-53da623065e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:50.719353 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2892573e-b3f3-4b3c-aad7-53da623065e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:50.720721 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 259/524] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Sep 14 00:51:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:50.721089 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2892573e-b3f3-4b3c-aad7-53da623065e1 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:50.721438 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86320621-02a9-4ba8-89d1-729d93df1327 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:50.739175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-2892573e-b3f3-4b3c-aad7-53da623065e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:50.739175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:50.739175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:50.739175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:50.739175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:50.739175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:50.739175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:50.739175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:50.739175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:50.739175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:50.739175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:50.739175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:50.739175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:51:50.739175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:51:50.739175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:51:50.739175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:51:50.740146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:50.740146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:50.740146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:50.740146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:50.740146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:50.740146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:50.740146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:50.740146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:50.740146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:50.740146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:50.740146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:50.740146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:50.740146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:50.740146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:50.740146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:50.740146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:50.740146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:51:50.741074 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:51:50.741074 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:50.741074 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:50.741074 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 266/525] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:50.742096 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892573e-b3f3-4b3c-aad7-53da623065e1 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:50.796496 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:50.800177 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:50.850627 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86320621-02a9-4ba8-89d1-729d93df1327 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:50.893099 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892573e-b3f3-4b3c-aad7-53da623065e1 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:50.894744 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e4354-d52d-4546-aad8-0fe5e17900ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:50.896449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e4354-d52d-4546-aad8-0fe5e17900ad None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:50.896977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e4354-d52d-4546-aad8-0fe5e17900ad None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:50.897460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e4354-d52d-4546-aad8-0fe5e17900ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:50.929396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3b6bf6-4886-4e48-af40-d170d1eb3ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:50.930018 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b6bf6-4886-4e48-af40-d170d1eb3ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:50.930301 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b6bf6-4886-4e48-af40-d170d1eb3ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:50.930576 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b6bf6-4886-4e48-af40-d170d1eb3ad9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:50.980795 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e4354-d52d-4546-aad8-0fe5e17900ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:50.982843 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e4354-d52d-4546-aad8-0fe5e17900ad None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:51.003361 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3b6bf6-4886-4e48-af40-d170d1eb3ad9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:51.004932 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3b6bf6-4886-4e48-af40-d170d1eb3ad9 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:51.041526 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:51.049954 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:51.104880 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 260/526] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:51:51.105948 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e4354-d52d-4546-aad8-0fe5e17900ad None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:51.126953 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 267/527] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:51:51.128176 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3b6bf6-4886-4e48-af40-d170d1eb3ad9 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:51.220030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e4354-d52d-4546-aad8-0fe5e17900ad None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:51.237950 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3b6bf6-4886-4e48-af40-d170d1eb3ad9 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:51.258017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b876878-3a1c-4e4c-a91a-83e1e70f75f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:51.258908 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b876878-3a1c-4e4c-a91a-83e1e70f75f6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:51.259375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b876878-3a1c-4e4c-a91a-83e1e70f75f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:51.259857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b876878-3a1c-4e4c-a91a-83e1e70f75f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:51.260672 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b876878-3a1c-4e4c-a91a-83e1e70f75f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:51.261172 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b876878-3a1c-4e4c-a91a-83e1e70f75f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:51.262754 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b876878-3a1c-4e4c-a91a-83e1e70f75f6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:51.276124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 261/528] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:51.277680 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b876878-3a1c-4e4c-a91a-83e1e70f75f6 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:51.283309 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca5f77-af99-49d4-990b-83f41eae65f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:51.284042 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca5f77-af99-49d4-990b-83f41eae65f5 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:51.284342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca5f77-af99-49d4-990b-83f41eae65f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:51.284663 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca5f77-af99-49d4-990b-83f41eae65f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:51.286781 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:51.298056 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ca5f77-af99-49d4-990b-83f41eae65f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:51.299788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:51.300654 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ca5f77-af99-49d4-990b-83f41eae65f5 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:51.319195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-51ca5f77-af99-49d4-990b-83f41eae65f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:51.319195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:51.319195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:51.319195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:51.319195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:51.319195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:51.319195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:51.319195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:51.319195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:51.319195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:51.319195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:51.319195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:51.319195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:51:51.319195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:51:51.319195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:51:51.319195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:51:51.320362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:51.320362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:51.320362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:51.320362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:51.320362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:51.320362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:51.320362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:51.320362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:51.320362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:51.320362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:51.320362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:51.320362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:51.320362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:51.320362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:51.320362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:51.320362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:51.320362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:51:51.321214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:51:51.321214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:51.321214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:51.325345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 268/529] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:51.326345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca5f77-af99-49d4-990b-83f41eae65f5 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:51.388842 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b876878-3a1c-4e4c-a91a-83e1e70f75f6 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:51.424419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b469ed-f6d0-4a15-a540-be99161efe2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:51.425516 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b469ed-f6d0-4a15-a540-be99161efe2b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:51.425999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b469ed-f6d0-4a15-a540-be99161efe2b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:51.426490 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b469ed-f6d0-4a15-a540-be99161efe2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:51.439533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b469ed-f6d0-4a15-a540-be99161efe2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:51.441380 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b469ed-f6d0-4a15-a540-be99161efe2b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:51.443526 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca5f77-af99-49d4-990b-83f41eae65f5 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:51.457529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-05b469ed-f6d0-4a15-a540-be99161efe2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:51.457529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:51.457529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:51.457529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:51.457529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:51.457529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:51.457529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:51.457529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:51.457529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:51.457529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:51.457529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:51.457529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:51.457529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:51:51.457529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:51:51.457529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:51:51.457529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:51:51.458527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:51.458527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:51.458527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:51.458527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:51.458527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:51.458527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:51.458527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:51.458527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:51.458527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:51.458527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:51.458527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:51.458527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:51.458527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:51.458527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:51.458527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:51.458527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:51.458527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:51:51.459370 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:51:51.459370 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:51.459370 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:51.461618 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 262/530] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:51.462720 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b469ed-f6d0-4a15-a540-be99161efe2b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:51.484089 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a058599-b59c-4605-9d3d-ef8320c76ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:51.485113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a058599-b59c-4605-9d3d-ef8320c76ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:51.485113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a058599-b59c-4605-9d3d-ef8320c76ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:51.485441 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a058599-b59c-4605-9d3d-ef8320c76ce3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:51.493169 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a058599-b59c-4605-9d3d-ef8320c76ce3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:51.494914 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a058599-b59c-4605-9d3d-ef8320c76ce3 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:51.508592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-5a058599-b59c-4605-9d3d-ef8320c76ce3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:51:51.508592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:51.508592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:51.508592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:51.508592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:51.508592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:51.508592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:51.508592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:51.508592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:51.508592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:51.508592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:51.508592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:51.508592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:51:51.508592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:51:51.508592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:51:51.509422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:51:51.509422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:51.509422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:51.509422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:51.509422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:51.509422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:51.509422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:51.509422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:51.509422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:51.509422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:51.509422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:51.509422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:51.509422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:51.509422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:51.509422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:51.509422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:51.509422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:51.510263 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:51:51.510263 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:51:51.510263 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:51:51.510263 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:51:51.510263 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:51:51.510263 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:51:51.510263 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:51:51.510263 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:51.510263 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 269/531] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:51:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:51.510729 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a058599-b59c-4605-9d3d-ef8320c76ce3 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:51.543743 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:51.568965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:51.578158 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b469ed-f6d0-4a15-a540-be99161efe2b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:51.614323 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad15f8b-f07d-4c02-accf-95b094c06e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:51.615274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad15f8b-f07d-4c02-accf-95b094c06e45 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:51.615795 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad15f8b-f07d-4c02-accf-95b094c06e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:51.616261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad15f8b-f07d-4c02-accf-95b094c06e45 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:51.617084 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad15f8b-f07d-4c02-accf-95b094c06e45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:51.617603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad15f8b-f07d-4c02-accf-95b094c06e45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:51.619666 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad15f8b-f07d-4c02-accf-95b094c06e45 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:51.630812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a058599-b59c-4605-9d3d-ef8320c76ce3 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:51.633256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 263/532] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:51.634285 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad15f8b-f07d-4c02-accf-95b094c06e45 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:51.674228 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ac86a-ccd8-47b7-a454-476429b38717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:51.674847 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ac86a-ccd8-47b7-a454-476429b38717 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:51.675130 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ac86a-ccd8-47b7-a454-476429b38717 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:51.675404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ac86a-ccd8-47b7-a454-476429b38717 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:51.676201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ac86a-ccd8-47b7-a454-476429b38717 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:51.676512 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ac86a-ccd8-47b7-a454-476429b38717 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:51.677896 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ac86a-ccd8-47b7-a454-476429b38717 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:51.691879 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 270/533] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:51.693015 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ac86a-ccd8-47b7-a454-476429b38717 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:51.739356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad15f8b-f07d-4c02-accf-95b094c06e45 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:51.772904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:51.780439 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f87b8-8b1e-450a-8dea-14296f99080f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:51.781374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f87b8-8b1e-450a-8dea-14296f99080f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:51.782178 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f87b8-8b1e-450a-8dea-14296f99080f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:51.782652 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f87b8-8b1e-450a-8dea-14296f99080f None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:51.783538 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f87b8-8b1e-450a-8dea-14296f99080f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:51.784096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f87b8-8b1e-450a-8dea-14296f99080f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:51.785739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f87b8-8b1e-450a-8dea-14296f99080f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:51.805930 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 264/534] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:51:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:51.807077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f87b8-8b1e-450a-8dea-14296f99080f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:51.811490 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ac86a-ccd8-47b7-a454-476429b38717 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:51.819895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:51.849387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d15781-7250-44a2-9eeb-5cb66c63020a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:51.850024 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d15781-7250-44a2-9eeb-5cb66c63020a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:51.850312 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d15781-7250-44a2-9eeb-5cb66c63020a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:51.850597 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d15781-7250-44a2-9eeb-5cb66c63020a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:51.858071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d15781-7250-44a2-9eeb-5cb66c63020a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:51.859774 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d15781-7250-44a2-9eeb-5cb66c63020a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:51.872234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-94d15781-7250-44a2-9eeb-5cb66c63020a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:51:51.872234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:51.872234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:51.872234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:51.872234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:51.872234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:51.872234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:51.872234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:51.872234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:51.872234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:51.872234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:51.872234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:51.872234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:51:51.872234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:51:51.872234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:51:51.873113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:51:51.873113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:51.873113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:51.873113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:51.873113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:51.873113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:51.873113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:51.873113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:51.873113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:51.873113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:51.873113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:51.873113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:51.873113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:51.873113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:51.873113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:51.873113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:51.873113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:51.873963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:51:51.873963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:51:51.873963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:51:51.873963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:51:51.873963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:51:51.873963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:51:51.873963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:51:51.873963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:51.878644 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 271/535] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:51:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:51.879537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d15781-7250-44a2-9eeb-5cb66c63020a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:51.919770 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f87b8-8b1e-450a-8dea-14296f99080f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:51.954591 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238e1c7-c121-4c7b-a90f-e53f5ddb1490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:51.955603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238e1c7-c121-4c7b-a90f-e53f5ddb1490 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:51.956099 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238e1c7-c121-4c7b-a90f-e53f5ddb1490 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:51.956566 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238e1c7-c121-4c7b-a90f-e53f5ddb1490 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:51.963401 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238e1c7-c121-4c7b-a90f-e53f5ddb1490 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:51.965138 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238e1c7-c121-4c7b-a90f-e53f5ddb1490 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:51.977115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-d238e1c7-c121-4c7b-a90f-e53f5ddb1490 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:51:51.977115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:51.977115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:51.977115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:51.977115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:51.977115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:51.977115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:51.977115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:51.977115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:51.977115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:51.977115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:51.977115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:51.977115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:51:51.977115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:51:51.977115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:51:51.978057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:51:51.978057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:51.978057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:51.978057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:51.978057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:51.978057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:51.978057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:51.978057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:51.978057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:51.978057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:51.978057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:51.978057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:51.978057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:51.978057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:51.978057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:51.978057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:51.978057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:51.978898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:51:51.978898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:51:51.978898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:51:51.978898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:51:51.978898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:51:51.978898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:51:51.978898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:51:51.978898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:51.980893 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 265/536] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:51:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:51.981855 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238e1c7-c121-4c7b-a90f-e53f5ddb1490 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:51.986550 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d15781-7250-44a2-9eeb-5cb66c63020a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:52.023104 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea1d37-4d6f-4093-b703-eb84ba22d352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:52.023692 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea1d37-4d6f-4093-b703-eb84ba22d352 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:52.023979 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea1d37-4d6f-4093-b703-eb84ba22d352 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:52.024255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea1d37-4d6f-4093-b703-eb84ba22d352 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:52.024975 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ea1d37-4d6f-4093-b703-eb84ba22d352 None admin] RBAC: Adding query filter params (role.id=7a33c2ceb92a478f81753573f8ece1bd, scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3, user.id=89ff357209194276bc079d0c05898747) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:52.025286 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ea1d37-4d6f-4093-b703-eb84ba22d352 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:52.026728 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ea1d37-4d6f-4093-b703-eb84ba22d352 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:52.039709 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 272/537] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:51:51 2022] GET /identity/v3/role_assignments?user.id=89ff357209194276bc079d0c05898747&scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3&role.id=7a33c2ceb92a478f81753573f8ece1bd => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:52.040056 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:52.040723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea1d37-4d6f-4093-b703-eb84ba22d352 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:52.091933 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238e1c7-c121-4c7b-a90f-e53f5ddb1490 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:52.098860 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:52.147827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d39a5b-cc3d-4d34-95c7-21ad432fa354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:52.150881 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d39a5b-cc3d-4d34-95c7-21ad432fa354 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:52.151479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d39a5b-cc3d-4d34-95c7-21ad432fa354 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:52.151982 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d39a5b-cc3d-4d34-95c7-21ad432fa354 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:52.152990 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d39a5b-cc3d-4d34-95c7-21ad432fa354 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:52.153834 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d39a5b-cc3d-4d34-95c7-21ad432fa354 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:52.155535 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d39a5b-cc3d-4d34-95c7-21ad432fa354 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:52.186743 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 266/538] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:51:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:52.191955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d39a5b-cc3d-4d34-95c7-21ad432fa354 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:52.267149 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea1d37-4d6f-4093-b703-eb84ba22d352 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:52.306815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc3905-104a-4f80-8b15-47f62dee6449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:52.306999 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc3905-104a-4f80-8b15-47f62dee6449 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:52.307219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc3905-104a-4f80-8b15-47f62dee6449 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:52.307505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc3905-104a-4f80-8b15-47f62dee6449 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:52.308242 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc3905-104a-4f80-8b15-47f62dee6449 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:52.309697 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc3905-104a-4f80-8b15-47f62dee6449 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:52.311478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc3905-104a-4f80-8b15-47f62dee6449 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:52.332874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 273/539] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:51:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:52.338895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc3905-104a-4f80-8b15-47f62dee6449 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:52.373186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d39a5b-cc3d-4d34-95c7-21ad432fa354 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:52.386344 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:52.418821 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b4fb2e-6d9d-497b-981b-e9da95159641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:52.419957 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4fb2e-6d9d-497b-981b-e9da95159641 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:52.420418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4fb2e-6d9d-497b-981b-e9da95159641 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:52.421253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b4fb2e-6d9d-497b-981b-e9da95159641 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:52.422361 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4fb2e-6d9d-497b-981b-e9da95159641 None admin] RBAC: Adding query filter params (role.id=7a33c2ceb92a478f81753573f8ece1bd, scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3, user.id=4d64f9bca7c546e4ba7f54c1c47d665e) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:52.423744 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4fb2e-6d9d-497b-981b-e9da95159641 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:52.425368 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b4fb2e-6d9d-497b-981b-e9da95159641 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:52.436787 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 267/540] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:51:52 2022] GET /identity/v3/role_assignments?user.id=4d64f9bca7c546e4ba7f54c1c47d665e&scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3&role.id=7a33c2ceb92a478f81753573f8ece1bd => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:52.557968 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:52.574151 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc3905-104a-4f80-8b15-47f62dee6449 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:52.630953 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f0cf5-a1a7-4948-9d0f-3f532364bc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:52.631585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f0cf5-a1a7-4948-9d0f-3f532364bc4a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:52.631878 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f0cf5-a1a7-4948-9d0f-3f532364bc4a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:52.632147 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f0cf5-a1a7-4948-9d0f-3f532364bc4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:52.632873 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f0cf5-a1a7-4948-9d0f-3f532364bc4a None admin] RBAC: Adding query filter params (role.id=7a33c2ceb92a478f81753573f8ece1bd, scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3, user.id=9373aa297439487691fae325c0245bb9) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:52.633176 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f0cf5-a1a7-4948-9d0f-3f532364bc4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:52.634604 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f0cf5-a1a7-4948-9d0f-3f532364bc4a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:52.644055 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 274/541] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:51:52 2022] GET /identity/v3/role_assignments?user.id=9373aa297439487691fae325c0245bb9&scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3&role.id=7a33c2ceb92a478f81753573f8ece1bd => generated 259 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:52.848815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8c143-18b4-46a3-9b9a-26e263907de7 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:52.849308 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8c143-18b4-46a3-9b9a-26e263907de7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:52.849384 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8c143-18b4-46a3-9b9a-26e263907de7 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:52.850903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 268/542] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:52.856723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bff935-5744-44d6-8943-3925c7d119d5 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:52.857046 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bff935-5744-44d6-8943-3925c7d119d5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:52.857277 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bff935-5744-44d6-8943-3925c7d119d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:52.863144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a41b9d-0d56-4881-9401-b196218204d5 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:52.863452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a41b9d-0d56-4881-9401-b196218204d5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:52.863817 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a41b9d-0d56-4881-9401-b196218204d5 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:52.865505 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 269/543] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:52.870594 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c9365-a812-40a2-a7a7-d681d5429076 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:52.870866 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c9365-a812-40a2-a7a7-d681d5429076 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:52.871140 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c9365-a812-40a2-a7a7-d681d5429076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:52.896584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-80bff935-5744-44d6-8943-3925c7d119d5 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:52.897795 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-80bff935-5744-44d6-8943-3925c7d119d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:52.906244 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7d3c9365-a812-40a2-a7a7-d681d5429076 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:52.908296 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7d3c9365-a812-40a2-a7a7-d681d5429076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:53.014301 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 275/544] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:51:53.034946 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 270/545] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:51:53.315506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bff935-5744-44d6-8943-3925c7d119d5 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:53.383568 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c9365-a812-40a2-a7a7-d681d5429076 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:53.459992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:53.500052 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bff935-5744-44d6-8943-3925c7d119d5 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:53.536141 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c9365-a812-40a2-a7a7-d681d5429076 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:53.538575 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:53.572585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e18034-bb2a-409a-b392-a0a30cf58ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:53.573205 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e18034-bb2a-409a-b392-a0a30cf58ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:53.573436 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e18034-bb2a-409a-b392-a0a30cf58ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:53.573715 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e18034-bb2a-409a-b392-a0a30cf58ae2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:53.585066 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdfda0e-3d79-414b-92dd-b8d0e5f57ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:53.585421 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e18034-bb2a-409a-b392-a0a30cf58ae2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:53.585709 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfda0e-3d79-414b-92dd-b8d0e5f57ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:53.585989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfda0e-3d79-414b-92dd-b8d0e5f57ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:53.586258 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfda0e-3d79-414b-92dd-b8d0e5f57ad5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:53.590516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-d6e18034-bb2a-409a-b392-a0a30cf58ae2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:51:53.590516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:53.590516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:53.590516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:53.590516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:53.590516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:53.590516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:53.590516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:53.590516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:53.590516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:53.590516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:53.590516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:53.590516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:53.590516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:53.590516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:53.590516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:53.591388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:53.591388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:53.591388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:53.591388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:53.591388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:53.591388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:53.591388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:53.591388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:53.591388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:53.591388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:53.591388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:53.591388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:53.591388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:53.591388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:53.591388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:53.591388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:53.592235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:53.592235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:53.592235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:53.592235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:53.592235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:53.592235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:53.592235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:53.592235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:53.592235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:53.592235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:53.592235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:51:53.592235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:53.592235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 276/546] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 00:51:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:53.600759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdfda0e-3d79-414b-92dd-b8d0e5f57ad5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:53.601833 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e18034-bb2a-409a-b392-a0a30cf58ae2 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:53.606252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-acdfda0e-3d79-414b-92dd-b8d0e5f57ad5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:51:53.606252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:53.606252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:53.606252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:53.606252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:53.606252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:53.606252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:53.606252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:53.606252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:53.606252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:53.606252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:53.606252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:53.606252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:53.606252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:53.606252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:53.606252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:53.607347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:53.607347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:53.607347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:53.607347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:53.607347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:53.607347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:53.607347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:53.607347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:53.607347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:53.607347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:53.607347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:53.607347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:53.607347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:53.607347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:53.607347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:53.607347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:53.608399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:53.608399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:53.608399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:53.608399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:53.608399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:53.608399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:53.608399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:53.608399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:53.608399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:53.608399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:53.608399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:51:53.608399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:53.611031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 271/547] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 00:51:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:51:53.617872 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdfda0e-3d79-414b-92dd-b8d0e5f57ad5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:53.773923 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e18034-bb2a-409a-b392-a0a30cf58ae2 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:53.815579 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdfda0e-3d79-414b-92dd-b8d0e5f57ad5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:53.824827 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d98e1a-62e7-4331-a795-e7690501e97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:53.825692 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d98e1a-62e7-4331-a795-e7690501e97d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:53.826178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d98e1a-62e7-4331-a795-e7690501e97d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:53.826658 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d98e1a-62e7-4331-a795-e7690501e97d None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:53.827637 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d98e1a-62e7-4331-a795-e7690501e97d None admin] RBAC: Adding query filter params (name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:53.828150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d98e1a-62e7-4331-a795-e7690501e97d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:53.830827 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d98e1a-62e7-4331-a795-e7690501e97d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:53.839546 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 277/548] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 00:51:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:53.847051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d98e1a-62e7-4331-a795-e7690501e97d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:53.860621 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:53.865627 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8bf87-3b1b-4f0d-999f-2e6649dbed8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:53.866818 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8bf87-3b1b-4f0d-999f-2e6649dbed8b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:53.867301 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8bf87-3b1b-4f0d-999f-2e6649dbed8b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:53.867817 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8bf87-3b1b-4f0d-999f-2e6649dbed8b None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:53.868689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8bf87-3b1b-4f0d-999f-2e6649dbed8b None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:53.869498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8bf87-3b1b-4f0d-999f-2e6649dbed8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:53.871229 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e8bf87-3b1b-4f0d-999f-2e6649dbed8b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:53.881059 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 272/549] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 00:51:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:53.887714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8bf87-3b1b-4f0d-999f-2e6649dbed8b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:53.947928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:54.018448 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8bf87-3b1b-4f0d-999f-2e6649dbed8b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:54.041714 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d98e1a-62e7-4331-a795-e7690501e97d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:54.058373 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f9881-a0f9-43b4-879d-83d07a7b91ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:54.059340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f9881-a0f9-43b4-879d-83d07a7b91ca None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:54.059881 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f9881-a0f9-43b4-879d-83d07a7b91ca None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:54.060378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f9881-a0f9-43b4-879d-83d07a7b91ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:54.073394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f9881-a0f9-43b4-879d-83d07a7b91ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:54.075822 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f9881-a0f9-43b4-879d-83d07a7b91ca None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:54.094958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-1d5f9881-a0f9-43b4-879d-83d07a7b91ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:54.094958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:54.094958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:54.094958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:54.094958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:54.094958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:54.094958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:54.094958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:54.094958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:54.094958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:54.094958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:54.094958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:54.094958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:51:54.094958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:51:54.094958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:51:54.094958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:51:54.096156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:54.096156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:54.096156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:54.096156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:54.096156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:54.096156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:54.096156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:54.096156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:54.096156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:54.096156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:54.096156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:54.096156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:54.096156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:54.096156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:54.096156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:54.096156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:54.096156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:51:54.097023 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:51:54.097023 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:54.097023 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:54.099101 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 273/550] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:54.108052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f9881-a0f9-43b4-879d-83d07a7b91ca None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:54.122520 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3666a7-caeb-4ab7-82d0-9f5892ce08ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:54.123162 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3666a7-caeb-4ab7-82d0-9f5892ce08ba None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:54.123437 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3666a7-caeb-4ab7-82d0-9f5892ce08ba None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:54.123730 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3666a7-caeb-4ab7-82d0-9f5892ce08ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:54.141822 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3666a7-caeb-4ab7-82d0-9f5892ce08ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:54.143283 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3666a7-caeb-4ab7-82d0-9f5892ce08ba None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:54.170140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-5d3666a7-caeb-4ab7-82d0-9f5892ce08ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:54.170140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:54.170140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:54.170140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:54.170140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:54.170140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:54.170140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:54.170140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:54.170140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:54.170140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:54.170140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:54.170140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:54.170140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:51:54.170140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:51:54.170140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:51:54.170140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:51:54.171125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:54.171125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:54.171125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:54.171125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:54.171125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:54.171125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:54.171125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:54.171125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:54.171125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:54.171125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:54.171125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:54.171125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:54.171125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:54.171125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:54.171125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:54.171125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:54.171125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:51:54.172216 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:51:54.172216 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:54.172216 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:54.173833 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 278/551] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:51:54.178581 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3666a7-caeb-4ab7-82d0-9f5892ce08ba None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:54.271969 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:54.281634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:54.314727 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f9881-a0f9-43b4-879d-83d07a7b91ca None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:54.362097 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3666a7-caeb-4ab7-82d0-9f5892ce08ba None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:54.381206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db9ca9-7df3-4025-9762-87f6d80f029a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:54.381827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db9ca9-7df3-4025-9762-87f6d80f029a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:54.382110 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db9ca9-7df3-4025-9762-87f6d80f029a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:54.382389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db9ca9-7df3-4025-9762-87f6d80f029a None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:54.383036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0db9ca9-7df3-4025-9762-87f6d80f029a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:54.383345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0db9ca9-7df3-4025-9762-87f6d80f029a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:54.384755 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0db9ca9-7df3-4025-9762-87f6d80f029a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:54.410378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 274/552] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:54.414630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db9ca9-7df3-4025-9762-87f6d80f029a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:54.430068 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93128634-a9c0-4d5a-b1f1-7fc00d1ea80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:54.430696 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93128634-a9c0-4d5a-b1f1-7fc00d1ea80d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:54.430980 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93128634-a9c0-4d5a-b1f1-7fc00d1ea80d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:54.431261 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93128634-a9c0-4d5a-b1f1-7fc00d1ea80d None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:54.431950 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93128634-a9c0-4d5a-b1f1-7fc00d1ea80d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:54.432258 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93128634-a9c0-4d5a-b1f1-7fc00d1ea80d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:54.433650 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93128634-a9c0-4d5a-b1f1-7fc00d1ea80d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:54.452988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 279/553] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:54.459241 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93128634-a9c0-4d5a-b1f1-7fc00d1ea80d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:54.538045 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db9ca9-7df3-4025-9762-87f6d80f029a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:54.576245 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681b809-e132-4e3d-a57a-0901005bb860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:54.576575 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681b809-e132-4e3d-a57a-0901005bb860 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:54.576800 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681b809-e132-4e3d-a57a-0901005bb860 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:54.577078 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681b809-e132-4e3d-a57a-0901005bb860 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:54.585020 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681b809-e132-4e3d-a57a-0901005bb860 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:54.590224 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681b809-e132-4e3d-a57a-0901005bb860 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:54.601479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-5681b809-e132-4e3d-a57a-0901005bb860 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 00:51:54.601479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:54.601479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:54.601479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:54.601479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:54.601479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:54.601479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:54.601479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:54.601479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:54.601479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:54.601479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:54.601479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:54.601479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:51:54.601479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:51:54.601479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:51:54.601479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:51:54.602547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:54.602547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:54.602547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:51:54.602547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:51:54.602547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:51:54.602547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:51:54.602547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:54.602547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:54.602547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:54.602547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:54.602547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:54.602547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:54.602547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:54.602547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:54.602547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:54.602547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:54.602547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:54.602547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:54.603514 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:54.603514 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:54.603514 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:51:54.603514 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:51:54.603514 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:51:54.603514 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:51:54.603514 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:51:54.603514 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:51:54.603514 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 00:51:54.603514 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:54.603514 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 275/554] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Sep 14 00:51:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:54.604120 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:54.610486 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681b809-e132-4e3d-a57a-0901005bb860 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:54.665163 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93128634-a9c0-4d5a-b1f1-7fc00d1ea80d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:54.689370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:54.716123 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1805b-7503-46a5-87f0-99b3a5716052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:54.717028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1805b-7503-46a5-87f0-99b3a5716052 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:54.718225 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1805b-7503-46a5-87f0-99b3a5716052 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:54.718516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1805b-7503-46a5-87f0-99b3a5716052 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:54.728223 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d1805b-7503-46a5-87f0-99b3a5716052 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:54.729972 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d1805b-7503-46a5-87f0-99b3a5716052 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:54.747438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-b1d1805b-7503-46a5-87f0-99b3a5716052 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 00:51:54.747438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:54.747438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:54.747438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:54.747438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:54.747438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:54.747438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:54.747438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:54.747438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:54.747438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:54.747438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:54.747438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:54.747438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:51:54.747438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:51:54.747438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:51:54.747438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:51:54.748537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:54.748537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:54.748537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:51:54.748537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:51:54.748537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:51:54.748537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:51:54.748537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:54.748537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:54.748537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:54.748537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:54.748537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:54.748537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:54.748537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:54.748537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:54.748537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:54.748537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:54.748537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:54.748537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:54.749913 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:54.749913 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:54.749913 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:51:54.749913 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:51:54.749913 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:51:54.749913 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:51:54.749913 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:51:54.749913 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:51:54.749913 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 00:51:54.749913 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:54.749913 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 280/555] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:51:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:54.755206 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6d776-2899-4ee4-96e4-79ec46fc8b53 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:54.755206 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6d776-2899-4ee4-96e4-79ec46fc8b53 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:54.755206 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6d776-2899-4ee4-96e4-79ec46fc8b53 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:54.755761 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681b809-e132-4e3d-a57a-0901005bb860 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:54.755927 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 281/556] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:54.756170 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6d776-2899-4ee4-96e4-79ec46fc8b53 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:54.794477 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9e092-a1eb-4358-b748-0dfa79bc85f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:54.795086 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9e092-a1eb-4358-b748-0dfa79bc85f1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:54.795364 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9e092-a1eb-4358-b748-0dfa79bc85f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:54.795678 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9e092-a1eb-4358-b748-0dfa79bc85f1 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:54.796400 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9e092-a1eb-4358-b748-0dfa79bc85f1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:54.796701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9e092-a1eb-4358-b748-0dfa79bc85f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:54.799119 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9e092-a1eb-4358-b748-0dfa79bc85f1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:54.817916 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 276/557] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Sep 14 00:51:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:51:54.819363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8223e2-1755-4646-90cb-30cc29c32d38 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:54.819687 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8223e2-1755-4646-90cb-30cc29c32d38 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:54.819967 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8223e2-1755-4646-90cb-30cc29c32d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:54.839361 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-8c8223e2-1755-4646-90cb-30cc29c32d38 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:54.840677 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8c8223e2-1755-4646-90cb-30cc29c32d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:54.896273 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6d776-2899-4ee4-96e4-79ec46fc8b53 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:54.901751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:54.927950 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 277/558] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:54.928944 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8223e2-1755-4646-90cb-30cc29c32d38 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:54.941653 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1da571-0c42-4f3b-be25-4649b4c2df67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:54.942281 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1da571-0c42-4f3b-be25-4649b4c2df67 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:54.942569 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1da571-0c42-4f3b-be25-4649b4c2df67 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:54.942848 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1da571-0c42-4f3b-be25-4649b4c2df67 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:54.943611 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1da571-0c42-4f3b-be25-4649b4c2df67 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:54.944026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1da571-0c42-4f3b-be25-4649b4c2df67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:54.946119 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1da571-0c42-4f3b-be25-4649b4c2df67 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:54.967720 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 282/559] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Sep 14 00:51:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:54.973999 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1da571-0c42-4f3b-be25-4649b4c2df67 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:55.053731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:55.123365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8223e2-1755-4646-90cb-30cc29c32d38 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:55.157863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1da571-0c42-4f3b-be25-4649b4c2df67 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:55.159638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d91dcd-0521-41e3-ab2d-3771c57da1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:55.160446 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d91dcd-0521-41e3-ab2d-3771c57da1bc None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:55.160726 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d91dcd-0521-41e3-ab2d-3771c57da1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:55.161022 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d91dcd-0521-41e3-ab2d-3771c57da1bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:55.171241 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d91dcd-0521-41e3-ab2d-3771c57da1bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:55.173345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d91dcd-0521-41e3-ab2d-3771c57da1bc None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:55.189994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-27d91dcd-0521-41e3-ab2d-3771c57da1bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:55.189994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:55.189994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:55.189994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:55.189994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:55.189994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:55.189994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:55.189994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:55.189994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:55.189994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:55.189994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:55.189994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:55.189994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:51:55.189994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:51:55.189994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:51:55.189994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:51:55.190936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:55.190936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:55.190936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:55.190936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:55.190936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:55.190936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:55.190936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:55.190936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:55.190936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:55.190936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:55.190936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:55.190936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:55.190936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:55.190936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:55.190936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:55.190936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:55.190936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:51:55.191848 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:51:55.191848 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:55.191848 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:55.191848 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 278/560] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:55.197407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b135090-e5fa-442f-86ba-c7eb354b3980 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:55.197687 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b135090-e5fa-442f-86ba-c7eb354b3980 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:55.197954 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b135090-e5fa-442f-86ba-c7eb354b3980 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:55.199338 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 279/561] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:55.201645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b135090-e5fa-442f-86ba-c7eb354b3980 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:55.211915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc982a8-9b1e-43de-a3c5-b569fbc46b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:55.212904 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc982a8-9b1e-43de-a3c5-b569fbc46b30 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:55.213771 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc982a8-9b1e-43de-a3c5-b569fbc46b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:55.214254 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc982a8-9b1e-43de-a3c5-b569fbc46b30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:55.253297 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:55.294895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc982a8-9b1e-43de-a3c5-b569fbc46b30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:55.296376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc982a8-9b1e-43de-a3c5-b569fbc46b30 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:55.323887 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b135090-e5fa-442f-86ba-c7eb354b3980 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:55.332440 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:55.360919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06926af-1449-40d6-9da5-ec651a6ad6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:55.361530 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06926af-1449-40d6-9da5-ec651a6ad6a2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:55.361813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06926af-1449-40d6-9da5-ec651a6ad6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:55.362091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06926af-1449-40d6-9da5-ec651a6ad6a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:55.362732 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06926af-1449-40d6-9da5-ec651a6ad6a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:55.363034 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06926af-1449-40d6-9da5-ec651a6ad6a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:55.364442 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06926af-1449-40d6-9da5-ec651a6ad6a2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:55.375376 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 280/562] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:55.377083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404be16-13ca-47b9-a4e6-01433a4b367b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:55.377377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404be16-13ca-47b9-a4e6-01433a4b367b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:55.377678 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404be16-13ca-47b9-a4e6-01433a4b367b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:55.395992 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-5404be16-13ca-47b9-a4e6-01433a4b367b None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:55.397190 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5404be16-13ca-47b9-a4e6-01433a4b367b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:55.403624 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 283/563] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:51:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:51:55.404323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc982a8-9b1e-43de-a3c5-b569fbc46b30 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:55.464389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 281/564] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:55.465550 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21659ba-f998-4b9a-b05b-fb1e912639e8 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:55.465825 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21659ba-f998-4b9a-b05b-fb1e912639e8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:55.466097 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21659ba-f998-4b9a-b05b-fb1e912639e8 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:55.467384 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 282/565] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:55.468203 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21659ba-f998-4b9a-b05b-fb1e912639e8 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:55.508095 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:55.511014 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc982a8-9b1e-43de-a3c5-b569fbc46b30 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:55.547491 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94e831-864e-4fec-92cf-ed26cec49a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:55.548426 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94e831-864e-4fec-92cf-ed26cec49a5c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:55.548894 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94e831-864e-4fec-92cf-ed26cec49a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:55.549350 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94e831-864e-4fec-92cf-ed26cec49a5c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:55.552495 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:55.561058 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94e831-864e-4fec-92cf-ed26cec49a5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:55.566198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-aa94e831-864e-4fec-92cf-ed26cec49a5c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:51:55.566198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:55.566198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:55.566198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:55.566198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:55.566198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:55.566198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:55.566198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:55.566198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:55.566198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:55.566198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:55.566198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:55.566198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:55.566198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:55.566198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:55.566198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:55.567125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:55.567125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:55.567125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:55.567125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:55.567125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:55.567125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:55.567125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:55.567125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:55.567125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:55.567125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:55.567125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:55.567125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:55.567125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:55.567125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:55.567125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:55.567125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:55.568022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:55.568022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:55.568022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:55.568022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:55.568022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:55.568022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:55.568022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:55.568022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:55.568022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:55.568022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:55.568022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:51:55.568022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:55.570157 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 284/566] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 00:51:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:55.571141 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94e831-864e-4fec-92cf-ed26cec49a5c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:55.592495 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21659ba-f998-4b9a-b05b-fb1e912639e8 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:55.634051 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9347be7-5192-4bfd-afe8-23b5e2b90342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:55.634964 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9347be7-5192-4bfd-afe8-23b5e2b90342 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:55.635247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9347be7-5192-4bfd-afe8-23b5e2b90342 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:55.635522 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9347be7-5192-4bfd-afe8-23b5e2b90342 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:55.643518 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9347be7-5192-4bfd-afe8-23b5e2b90342 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:55.645001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9347be7-5192-4bfd-afe8-23b5e2b90342 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:55.663253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-d9347be7-5192-4bfd-afe8-23b5e2b90342 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:51:55.663253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:55.663253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:55.663253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:55.663253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:55.663253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:55.663253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:55.663253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:55.663253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:55.663253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:55.663253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:55.663253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:55.663253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:51:55.663253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:51:55.663253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:51:55.664412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:51:55.664412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:55.664412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:55.664412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:55.664412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:55.664412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:55.664412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:55.664412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:55.664412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:55.664412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:55.664412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:55.664412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:55.664412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:55.664412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:55.664412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:55.664412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:55.664412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:55.665304 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:51:55.665304 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:51:55.665304 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:51:55.665304 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:51:55.665304 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:51:55.665304 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:51:55.665304 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:51:55.665304 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:55.665304 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 283/567] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:51:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:55.666307 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955faf53-7dbd-44a6-8770-e6426e3f1bc0 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:55.666307 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955faf53-7dbd-44a6-8770-e6426e3f1bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:55.666307 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955faf53-7dbd-44a6-8770-e6426e3f1bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:55.684006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-955faf53-7dbd-44a6-8770-e6426e3f1bc0 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:55.685184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-955faf53-7dbd-44a6-8770-e6426e3f1bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:55.687471 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94e831-864e-4fec-92cf-ed26cec49a5c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:55.723289 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea6198-b2cb-444f-a6f9-6f8daf97b3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:55.723862 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea6198-b2cb-444f-a6f9-6f8daf97b3d2 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:55.724144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea6198-b2cb-444f-a6f9-6f8daf97b3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:55.724415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea6198-b2cb-444f-a6f9-6f8daf97b3d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:55.735315 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea6198-b2cb-444f-a6f9-6f8daf97b3d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:55.736840 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea6198-b2cb-444f-a6f9-6f8daf97b3d2 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:55.751910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-7fea6198-b2cb-444f-a6f9-6f8daf97b3d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:55.751910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:55.751910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:55.751910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:55.751910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:55.751910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:55.751910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:55.751910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:55.751910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:55.751910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:55.751910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:55.751910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:55.751910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:51:55.751910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:51:55.751910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:51:55.751910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:51:55.752938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:55.752938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:55.752938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:55.752938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:55.752938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:55.752938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:55.752938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:55.752938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:55.752938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:55.752938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:55.752938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:55.752938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:55.752938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:55.752938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:55.752938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:55.752938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:55.752938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:51:55.753873 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:51:55.753873 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:55.753873 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:55.753873 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 285/568] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:55.754199 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea6198-b2cb-444f-a6f9-6f8daf97b3d2 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:55.762633 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 284/569] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:55.763466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955faf53-7dbd-44a6-8770-e6426e3f1bc0 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:55.786472 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:55.803846 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:55.861597 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea6198-b2cb-444f-a6f9-6f8daf97b3d2 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:55.874357 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955faf53-7dbd-44a6-8770-e6426e3f1bc0 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:55.897519 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39648be1-a456-4ee1-a851-4bbdd63ab4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:55.898130 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39648be1-a456-4ee1-a851-4bbdd63ab4c0 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:55.898414 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39648be1-a456-4ee1-a851-4bbdd63ab4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:55.898690 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39648be1-a456-4ee1-a851-4bbdd63ab4c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:55.899348 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39648be1-a456-4ee1-a851-4bbdd63ab4c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:55.899685 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39648be1-a456-4ee1-a851-4bbdd63ab4c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:55.901130 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39648be1-a456-4ee1-a851-4bbdd63ab4c0 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:55.914241 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 286/570] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 00:51:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:55.914241 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39648be1-a456-4ee1-a851-4bbdd63ab4c0 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:55.915964 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248ae4f-7605-4be9-8e71-0f6c8c81adfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:55.916579 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248ae4f-7605-4be9-8e71-0f6c8c81adfa None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:55.916858 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248ae4f-7605-4be9-8e71-0f6c8c81adfa None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:55.917131 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248ae4f-7605-4be9-8e71-0f6c8c81adfa None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:55.917823 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248ae4f-7605-4be9-8e71-0f6c8c81adfa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:55.918127 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248ae4f-7605-4be9-8e71-0f6c8c81adfa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:55.919584 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248ae4f-7605-4be9-8e71-0f6c8c81adfa None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:55.935958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 285/571] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:51:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:55.935958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248ae4f-7605-4be9-8e71-0f6c8c81adfa None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:56.017596 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:56.021734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39648be1-a456-4ee1-a851-4bbdd63ab4c0 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:56.033044 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:56.040895 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248ae4f-7605-4be9-8e71-0f6c8c81adfa None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:56.056957 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072421b-a7bc-4e00-a022-849c06064eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:56.057585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072421b-a7bc-4e00-a022-849c06064eef None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:56.057861 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072421b-a7bc-4e00-a022-849c06064eef None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:56.058129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072421b-a7bc-4e00-a022-849c06064eef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:56.068210 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d072421b-a7bc-4e00-a022-849c06064eef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:56.073319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-d072421b-a7bc-4e00-a022-849c06064eef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:51:56.073319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:56.073319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:56.073319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:56.073319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:56.073319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:56.073319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:56.073319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:56.073319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:56.073319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:56.073319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:56.073319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:56.073319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:56.073319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:56.073319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:56.073319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:56.074466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:56.074466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:56.074466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:56.074466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:56.074466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:56.074466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:56.074466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:56.074466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:56.074466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:56.074466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:56.074466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:56.074466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:56.074466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:56.074466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:56.074466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:56.074466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:56.075865 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:56.075865 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:56.075865 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:56.075865 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:56.075865 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:56.075865 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:56.075865 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:56.075865 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:56.075865 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:56.075865 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:56.075865 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:51:56.075865 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:56.075865 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 287/572] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 00:51:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:56.075865 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072421b-a7bc-4e00-a022-849c06064eef None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:56.076830 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c158fe-cc9f-4349-8f72-8bd74fc0bf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:56.077386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c158fe-cc9f-4349-8f72-8bd74fc0bf35 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:56.077672 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c158fe-cc9f-4349-8f72-8bd74fc0bf35 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:56.077947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c158fe-cc9f-4349-8f72-8bd74fc0bf35 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:56.079168 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c158fe-cc9f-4349-8f72-8bd74fc0bf35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:56.079874 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c158fe-cc9f-4349-8f72-8bd74fc0bf35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:56.081287 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c158fe-cc9f-4349-8f72-8bd74fc0bf35 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:56.093932 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 286/573] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:56.095307 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c158fe-cc9f-4349-8f72-8bd74fc0bf35 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:56.179020 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072421b-a7bc-4e00-a022-849c06064eef None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:56.195262 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c158fe-cc9f-4349-8f72-8bd74fc0bf35 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:56.212247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ff0f5-5226-4095-b90b-d697b57231c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:56.212876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ff0f5-5226-4095-b90b-d697b57231c8 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:56.213155 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ff0f5-5226-4095-b90b-d697b57231c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:56.213428 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ff0f5-5226-4095-b90b-d697b57231c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:56.224586 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ff0f5-5226-4095-b90b-d697b57231c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:56.226039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ff0f5-5226-4095-b90b-d697b57231c8 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:56.229522 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3b392-1cfe-40a7-a0b7-921f79c97d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:56.230183 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3b392-1cfe-40a7-a0b7-921f79c97d02 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:56.230469 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3b392-1cfe-40a7-a0b7-921f79c97d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:56.230747 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3b392-1cfe-40a7-a0b7-921f79c97d02 None admin] PATH_INFO: `/v3/projects/fda69e8c2d5e466dbc1244b2f76296a3/users/5e0fe2e84f6748c992c9beedf2f5d49b/roles/7a33c2ceb92a478f81753573f8ece1bd` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:56.238777 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:56.240219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-e63ff0f5-5226-4095-b90b-d697b57231c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:56.240219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:56.240219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:56.240219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:56.240219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:56.240219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:56.240219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:56.240219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:56.240219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:56.240219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:56.240219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:56.240219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:56.240219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:51:56.240219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:51:56.240219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:51:56.240219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:51:56.241308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:56.241308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:56.241308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:56.241308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:56.241308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:56.241308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:56.241308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:56.241308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:56.241308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:56.241308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:56.241308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:56.241308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:56.241308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:56.241308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:56.241308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:56.241308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:56.241308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:51:56.242201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:51:56.242201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:56.242201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:56.246253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 288/574] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:56.246621 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3b392-1cfe-40a7-a0b7-921f79c97d02 None admin] RBAC: Authorizing `identity:create_grant(project_id=fda69e8c2d5e466dbc1244b2f76296a3, user_id=5e0fe2e84f6748c992c9beedf2f5d49b, role_id=7a33c2ceb92a478f81753573f8ece1bd)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:56.247140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ff0f5-5226-4095-b90b-d697b57231c8 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:56.248205 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3b392-1cfe-40a7-a0b7-921f79c97d02 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:56.262052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 287/575] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 00:51:56 2022] PUT /identity/v3/projects/fda69e8c2d5e466dbc1244b2f76296a3/users/5e0fe2e84f6748c992c9beedf2f5d49b/roles/7a33c2ceb92a478f81753573f8ece1bd => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:51:56.263022 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3b392-1cfe-40a7-a0b7-921f79c97d02 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:56.274812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:56.368627 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ff0f5-5226-4095-b90b-d697b57231c8 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:56.403630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3b392-1cfe-40a7-a0b7-921f79c97d02 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:56.409119 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31910b64-44a3-48c0-b721-1b313dd1c68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:56.409838 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31910b64-44a3-48c0-b721-1b313dd1c68b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:56.410126 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31910b64-44a3-48c0-b721-1b313dd1c68b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:56.410402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31910b64-44a3-48c0-b721-1b313dd1c68b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:56.423441 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31910b64-44a3-48c0-b721-1b313dd1c68b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:56.429030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-31910b64-44a3-48c0-b721-1b313dd1c68b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:51:56.429030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:56.429030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:56.429030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:56.429030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:56.429030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:56.429030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:56.429030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:56.429030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:56.429030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:56.429030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:56.429030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:56.429030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:56.429030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:56.429030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:56.429030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:56.429959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:56.429959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:56.429959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:56.429959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:56.429959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:56.429959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:56.429959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:56.429959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:56.429959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:56.429959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:56.429959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:56.429959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:56.429959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:56.429959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:56.429959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:56.429959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:56.430812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:56.430812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:56.430812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:56.430812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:56.430812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:56.430812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:56.430812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:56.430812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:56.430812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:56.430812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:56.430812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:51:56.430812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:56.430812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 289/576] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 00:51:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:56.431539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31910b64-44a3-48c0-b721-1b313dd1c68b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:56.451965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e536b4bf-0ab3-4a42-9d06-1e6ba390827b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:56.452635 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536b4bf-0ab3-4a42-9d06-1e6ba390827b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:56.452915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536b4bf-0ab3-4a42-9d06-1e6ba390827b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:56.453191 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536b4bf-0ab3-4a42-9d06-1e6ba390827b None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:56.453893 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e536b4bf-0ab3-4a42-9d06-1e6ba390827b None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:56.454197 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e536b4bf-0ab3-4a42-9d06-1e6ba390827b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:56.455669 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e536b4bf-0ab3-4a42-9d06-1e6ba390827b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:56.465468 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 288/577] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 00:51:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:56.466299 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e536b4bf-0ab3-4a42-9d06-1e6ba390827b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:56.525161 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:56.528098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:56.548263 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31910b64-44a3-48c0-b721-1b313dd1c68b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:56.584070 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e536b4bf-0ab3-4a42-9d06-1e6ba390827b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:56.585412 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093753f-02cc-49d4-ad3a-b04ecbb5968b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:56.586060 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093753f-02cc-49d4-ad3a-b04ecbb5968b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:56.586395 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093753f-02cc-49d4-ad3a-b04ecbb5968b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:56.586679 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093753f-02cc-49d4-ad3a-b04ecbb5968b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:56.595023 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093753f-02cc-49d4-ad3a-b04ecbb5968b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:56.596564 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093753f-02cc-49d4-ad3a-b04ecbb5968b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:56.608105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-4093753f-02cc-49d4-ad3a-b04ecbb5968b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:51:56.608105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:56.608105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:56.608105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:56.608105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:56.608105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:56.608105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:56.608105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:56.608105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:56.608105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:56.608105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:56.608105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:56.608105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:51:56.608105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:51:56.608105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:51:56.609027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:51:56.609027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:56.609027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:56.609027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:56.609027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:56.609027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:56.609027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:56.609027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:56.609027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:56.609027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:56.609027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:56.609027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:56.609027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:56.609027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:56.609027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:56.609027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:56.609027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:56.609985 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:51:56.609985 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:51:56.609985 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:51:56.609985 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:51:56.609985 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:51:56.609985 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:51:56.609985 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:51:56.609985 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:56.609985 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 290/578] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:51:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:56.611027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093753f-02cc-49d4-ad3a-b04ecbb5968b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:56.627396 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b00eab-4372-4b92-aab6-45ebab917ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:56.628051 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b00eab-4372-4b92-aab6-45ebab917ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:56.628339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b00eab-4372-4b92-aab6-45ebab917ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:56.628617 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b00eab-4372-4b92-aab6-45ebab917ee2 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:56.629263 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b00eab-4372-4b92-aab6-45ebab917ee2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:56.629573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b00eab-4372-4b92-aab6-45ebab917ee2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:56.630964 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b00eab-4372-4b92-aab6-45ebab917ee2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:56.648950 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 289/579] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:56.648950 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b00eab-4372-4b92-aab6-45ebab917ee2 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:56.726392 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093753f-02cc-49d4-ad3a-b04ecbb5968b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:56.760566 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b00eab-4372-4b92-aab6-45ebab917ee2 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:56.762871 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34c873-4490-41e7-85b6-7dfce30054be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:56.763498 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34c873-4490-41e7-85b6-7dfce30054be None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:56.763800 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34c873-4490-41e7-85b6-7dfce30054be None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:56.764081 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34c873-4490-41e7-85b6-7dfce30054be None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:56.764744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf34c873-4490-41e7-85b6-7dfce30054be None admin] RBAC: Adding query filter params (name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:56.765048 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf34c873-4490-41e7-85b6-7dfce30054be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:56.766808 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf34c873-4490-41e7-85b6-7dfce30054be None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:56.773798 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:56.779707 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 291/580] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 00:51:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:56.779707 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34c873-4490-41e7-85b6-7dfce30054be None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:56.782392 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:56.805258 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fccf1-33f5-40c9-adc0-55df9e0aedad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:56.806004 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fccf1-33f5-40c9-adc0-55df9e0aedad None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:56.806187 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fccf1-33f5-40c9-adc0-55df9e0aedad None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:56.806440 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fccf1-33f5-40c9-adc0-55df9e0aedad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:56.816716 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85fccf1-33f5-40c9-adc0-55df9e0aedad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:56.818192 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85fccf1-33f5-40c9-adc0-55df9e0aedad None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:56.836533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-f85fccf1-33f5-40c9-adc0-55df9e0aedad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:56.836533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:56.836533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:56.836533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:56.836533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:56.836533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:56.836533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:56.836533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:56.836533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:56.836533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:56.836533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:56.836533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:56.836533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:51:56.836533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:51:56.836533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:51:56.836533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:51:56.837583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:56.837583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:56.837583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:56.837583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:56.837583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:56.837583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:56.837583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:56.837583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:56.837583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:56.837583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:56.837583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:56.837583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:56.837583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:56.837583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:56.837583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:56.837583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:56.837583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:51:56.838485 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:51:56.838485 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:56.838485 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:56.845199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 290/581] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:56.845199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fccf1-33f5-40c9-adc0-55df9e0aedad None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:56.894910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34c873-4490-41e7-85b6-7dfce30054be None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:56.930827 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e993ab2-9661-4ca2-9c3c-edf54c5a3391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:56.931769 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e993ab2-9661-4ca2-9c3c-edf54c5a3391 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:56.932057 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e993ab2-9661-4ca2-9c3c-edf54c5a3391 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:56.932336 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e993ab2-9661-4ca2-9c3c-edf54c5a3391 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:56.933032 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e993ab2-9661-4ca2-9c3c-edf54c5a3391 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:56.933345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e993ab2-9661-4ca2-9c3c-edf54c5a3391 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:56.935365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e993ab2-9661-4ca2-9c3c-edf54c5a3391 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:56.946496 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fccf1-33f5-40c9-adc0-55df9e0aedad None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:56.957756 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 292/582] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:51:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:56.957756 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e993ab2-9661-4ca2-9c3c-edf54c5a3391 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:56.984038 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3847f-7a47-405f-b9e4-9f26f5d15a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:56.984689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3847f-7a47-405f-b9e4-9f26f5d15a8d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:56.985513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3847f-7a47-405f-b9e4-9f26f5d15a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:56.985796 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3847f-7a47-405f-b9e4-9f26f5d15a8d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:56.994285 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d3847f-7a47-405f-b9e4-9f26f5d15a8d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:56.995775 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d3847f-7a47-405f-b9e4-9f26f5d15a8d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:57.007449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-d6d3847f-7a47-405f-b9e4-9f26f5d15a8d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 14 00:51:57.007449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:57.007449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:57.007449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:57.007449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:57.007449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:57.007449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:57.007449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:57.007449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:57.007449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:57.007449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:57.007449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:57.007449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:51:57.007449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:51:57.007449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:51:57.007449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:51:57.008637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:57.008637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:57.008637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:51:57.008637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:51:57.008637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:51:57.008637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:51:57.008637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:57.008637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:57.008637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:57.008637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:57.008637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:57.008637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:57.008637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:57.008637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:57.008637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:57.008637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:57.008637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:57.008637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:57.009616 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:57.009616 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:57.009616 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:51:57.009616 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:51:57.009616 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:51:57.009616 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:51:57.009616 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:51:57.009616 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:51:57.009616 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 14 00:51:57.009616 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:57.009616 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 291/583] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:57.010205 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3847f-7a47-405f-b9e4-9f26f5d15a8d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:57.028759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:57.042404 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:57.069521 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e993ab2-9661-4ca2-9c3c-edf54c5a3391 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:57.105429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604993a-c2ae-4ef8-9255-27a84894afde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:57.106060 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604993a-c2ae-4ef8-9255-27a84894afde None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:57.106341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604993a-c2ae-4ef8-9255-27a84894afde None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:57.106612 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604993a-c2ae-4ef8-9255-27a84894afde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:57.118169 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604993a-c2ae-4ef8-9255-27a84894afde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:57.119663 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604993a-c2ae-4ef8-9255-27a84894afde None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:57.125995 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3847f-7a47-405f-b9e4-9f26f5d15a8d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:57.135510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-2604993a-c2ae-4ef8-9255-27a84894afde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:57.135510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:57.135510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:57.135510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:57.135510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:57.135510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:57.135510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:57.135510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:57.135510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:57.135510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:57.135510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:57.135510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:57.135510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:51:57.135510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:51:57.135510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:51:57.135510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:51:57.136744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:57.136744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:57.136744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:57.136744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:57.136744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:57.136744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:57.136744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:57.136744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:57.136744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:57.136744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:57.136744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:57.136744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:57.136744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:57.136744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:57.136744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:57.136744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:57.136744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:51:57.137706 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:51:57.137706 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:57.137706 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:57.137706 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 293/584] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:57.138362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604993a-c2ae-4ef8-9255-27a84894afde None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:57.168725 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb4608-72c7-4ab2-aec6-5327307e9030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:57.169344 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb4608-72c7-4ab2-aec6-5327307e9030 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:57.169632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb4608-72c7-4ab2-aec6-5327307e9030 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:57.169911 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb4608-72c7-4ab2-aec6-5327307e9030 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:57.170556 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb4608-72c7-4ab2-aec6-5327307e9030 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:57.170866 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb4608-72c7-4ab2-aec6-5327307e9030 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:57.172265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb4608-72c7-4ab2-aec6-5327307e9030 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:57.184789 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 292/585] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:57.185602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb4608-72c7-4ab2-aec6-5327307e9030 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:57.249105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604993a-c2ae-4ef8-9255-27a84894afde None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:57.267013 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:57.271627 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:57.287421 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e968746a-6d68-4cf2-9d40-d03045ddfd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:57.288116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968746a-6d68-4cf2-9d40-d03045ddfd59 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:57.288516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968746a-6d68-4cf2-9d40-d03045ddfd59 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:57.288791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968746a-6d68-4cf2-9d40-d03045ddfd59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:57.289634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e968746a-6d68-4cf2-9d40-d03045ddfd59 None admin] RBAC: Adding query filter params (role.id=7a33c2ceb92a478f81753573f8ece1bd, scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3, user.id=e52776d9b4894af9b85a24136a8255e5) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:57.289965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e968746a-6d68-4cf2-9d40-d03045ddfd59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:57.291384 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e968746a-6d68-4cf2-9d40-d03045ddfd59 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:57.292005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb4608-72c7-4ab2-aec6-5327307e9030 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:57.297308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 294/586] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:51:57 2022] GET /identity/v3/role_assignments?user.id=e52776d9b4894af9b85a24136a8255e5&scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3&role.id=7a33c2ceb92a478f81753573f8ece1bd => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:57.298101 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e968746a-6d68-4cf2-9d40-d03045ddfd59 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:57.345600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab00053-afdf-47a8-a27f-d286697be1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:57.346630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab00053-afdf-47a8-a27f-d286697be1a3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:57.347115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab00053-afdf-47a8-a27f-d286697be1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:57.347628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab00053-afdf-47a8-a27f-d286697be1a3 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:57.348592 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab00053-afdf-47a8-a27f-d286697be1a3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:57.349164 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab00053-afdf-47a8-a27f-d286697be1a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:57.350877 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab00053-afdf-47a8-a27f-d286697be1a3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:57.373611 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 293/587] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Sep 14 00:51:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:57.374985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab00053-afdf-47a8-a27f-d286697be1a3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:57.430396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e968746a-6d68-4cf2-9d40-d03045ddfd59 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:57.483256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814d622-fb8f-4068-b169-7a3d9c7d8eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:57.484031 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814d622-fb8f-4068-b169-7a3d9c7d8eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:57.484318 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814d622-fb8f-4068-b169-7a3d9c7d8eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:57.484598 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814d622-fb8f-4068-b169-7a3d9c7d8eb2 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:57.485591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814d622-fb8f-4068-b169-7a3d9c7d8eb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:57.485591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814d622-fb8f-4068-b169-7a3d9c7d8eb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:57.487164 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814d622-fb8f-4068-b169-7a3d9c7d8eb2 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:57.501686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 295/588] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:57.502564 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814d622-fb8f-4068-b169-7a3d9c7d8eb2 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:57.520098 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab00053-afdf-47a8-a27f-d286697be1a3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:57.548415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:57.559405 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbce5ac-2f45-47cd-a1a6-43803127ede9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:57.560165 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbce5ac-2f45-47cd-a1a6-43803127ede9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:57.560450 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbce5ac-2f45-47cd-a1a6-43803127ede9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:57.560726 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbce5ac-2f45-47cd-a1a6-43803127ede9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:57.568849 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbce5ac-2f45-47cd-a1a6-43803127ede9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:57.570417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbce5ac-2f45-47cd-a1a6-43803127ede9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:57.573291 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:57.584209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-8dbce5ac-2f45-47cd-a1a6-43803127ede9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 00:51:57.584209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:57.584209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:57.584209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:57.584209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:57.584209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:57.584209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:57.584209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:57.584209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:57.584209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:57.584209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:57.584209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:57.584209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:51:57.584209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:51:57.584209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:51:57.584209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:51:57.585424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:57.585424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:57.585424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:51:57.585424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:51:57.585424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:51:57.585424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:51:57.585424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:57.585424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:57.585424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:57.585424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:57.585424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:57.585424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:57.585424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:57.585424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:57.585424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:57.585424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:57.585424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:57.585424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:57.586632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:57.586632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:57.586632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:51:57.586632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:51:57.586632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:51:57.586632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:51:57.586632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:51:57.586632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:51:57.586632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 00:51:57.586632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:57.592417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 294/589] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 00:51:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:57.593218 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbce5ac-2f45-47cd-a1a6-43803127ede9 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:57.624157 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814d622-fb8f-4068-b169-7a3d9c7d8eb2 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:57.666170 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9da85-2be1-4c6f-b140-b7ac92d76682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:57.666868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9da85-2be1-4c6f-b140-b7ac92d76682 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:57.667215 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9da85-2be1-4c6f-b140-b7ac92d76682 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:57.667493 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9da85-2be1-4c6f-b140-b7ac92d76682 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:57.678708 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade9da85-2be1-4c6f-b140-b7ac92d76682 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:57.680353 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade9da85-2be1-4c6f-b140-b7ac92d76682 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:57.695988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-ade9da85-2be1-4c6f-b140-b7ac92d76682 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:57.695988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:57.695988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:57.695988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:57.695988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:57.695988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:57.695988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:57.695988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:57.695988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:57.695988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:57.695988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:57.695988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:57.695988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:51:57.695988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:51:57.695988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:51:57.695988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:51:57.696982 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:57.696982 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:57.696982 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:57.696982 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:57.696982 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:57.696982 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:57.696982 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:57.696982 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:57.696982 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:57.696982 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:57.696982 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:57.696982 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:57.696982 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:57.696982 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:57.696982 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:57.696982 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:57.696982 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:51:57.697879 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:51:57.697879 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:57.697879 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:57.701753 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 296/590] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:57.702024 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbce5ac-2f45-47cd-a1a6-43803127ede9 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:57.702687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9da85-2be1-4c6f-b140-b7ac92d76682 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:57.737704 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ff150-ab5b-4468-a936-fec907005ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:57.738341 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ff150-ab5b-4468-a936-fec907005ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:57.738630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ff150-ab5b-4468-a936-fec907005ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:57.738908 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ff150-ab5b-4468-a936-fec907005ae0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:57.746548 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ff150-ab5b-4468-a936-fec907005ae0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:57.748033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045ff150-ab5b-4468-a936-fec907005ae0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:57.759913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-045ff150-ab5b-4468-a936-fec907005ae0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 00:51:57.759913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:57.759913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:57.759913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:57.759913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:57.759913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:57.759913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:57.759913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:57.759913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:57.759913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:57.759913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:57.759913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:57.759913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:51:57.759913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:51:57.759913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:51:57.759913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:51:57.762091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:57.762091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:57.762091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:51:57.762091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:51:57.762091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:51:57.762091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:51:57.762091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:57.762091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:57.762091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:57.762091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:57.762091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:57.762091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:57.762091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:57.762091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:57.762091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:57.762091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:57.762091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:57.762091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:57.764356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:57.764356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:57.764356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:51:57.764356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:51:57.764356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:51:57.764356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:51:57.764356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:51:57.764356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:51:57.764356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 00:51:57.764356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:57.765906 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 295/591] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:51:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:57.775593 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ff150-ab5b-4468-a936-fec907005ae0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:57.802164 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:57.813612 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9da85-2be1-4c6f-b140-b7ac92d76682 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:57.828501 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:57.852645 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5960b7-5624-4366-b6bf-96d4ea1c326a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:57.853590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5960b7-5624-4366-b6bf-96d4ea1c326a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:57.854012 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5960b7-5624-4366-b6bf-96d4ea1c326a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:57.854467 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5960b7-5624-4366-b6bf-96d4ea1c326a None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:57.855335 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5960b7-5624-4366-b6bf-96d4ea1c326a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:57.855859 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5960b7-5624-4366-b6bf-96d4ea1c326a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:57.857544 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5960b7-5624-4366-b6bf-96d4ea1c326a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:57.872624 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045ff150-ab5b-4468-a936-fec907005ae0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:57.877731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 297/592] 10.222.0.42 () {56 vars in 1139 bytes} [Wed Sep 14 00:51:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:57.878736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5960b7-5624-4366-b6bf-96d4ea1c326a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:57.908286 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b0343-d93c-4cec-a4c3-e70ed2d6404e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:57.908898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b0343-d93c-4cec-a4c3-e70ed2d6404e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:57.909183 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b0343-d93c-4cec-a4c3-e70ed2d6404e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:57.909460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b0343-d93c-4cec-a4c3-e70ed2d6404e None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:57.910265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b0343-d93c-4cec-a4c3-e70ed2d6404e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:57.910474 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b0343-d93c-4cec-a4c3-e70ed2d6404e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:57.911879 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b0343-d93c-4cec-a4c3-e70ed2d6404e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:57.928513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 296/593] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:57.928513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b0343-d93c-4cec-a4c3-e70ed2d6404e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:57.994266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5960b7-5624-4366-b6bf-96d4ea1c326a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:58.032194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b0343-d93c-4cec-a4c3-e70ed2d6404e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:58.033311 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb18d99e-9665-433a-b50c-01b1c32bb740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:58.034227 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18d99e-9665-433a-b50c-01b1c32bb740 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:58.034748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18d99e-9665-433a-b50c-01b1c32bb740 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:58.035201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18d99e-9665-433a-b50c-01b1c32bb740 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:58.036142 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb18d99e-9665-433a-b50c-01b1c32bb740 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:58.036624 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb18d99e-9665-433a-b50c-01b1c32bb740 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:58.038260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb18d99e-9665-433a-b50c-01b1c32bb740 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:58.054663 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:58.060605 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 298/594] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Sep 14 00:51:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:58.061738 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb18d99e-9665-433a-b50c-01b1c32bb740 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:58.075468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:58.075664 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d05be-899f-4adb-a457-89eb7cda0370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:58.076463 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d05be-899f-4adb-a457-89eb7cda0370 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:58.076573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d05be-899f-4adb-a457-89eb7cda0370 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:58.077098 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d05be-899f-4adb-a457-89eb7cda0370 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:58.088532 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d05be-899f-4adb-a457-89eb7cda0370 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:58.090025 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d05be-899f-4adb-a457-89eb7cda0370 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:58.108683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-273d05be-899f-4adb-a457-89eb7cda0370 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:58.108683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:58.108683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:58.108683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:58.108683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:58.108683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:58.108683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:58.108683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:58.108683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:58.108683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:58.108683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:58.108683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:58.108683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:51:58.108683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:51:58.108683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:51:58.108683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:51:58.109701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:58.109701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:58.109701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:58.109701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:58.109701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:58.109701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:58.109701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:58.109701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:58.109701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:58.109701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:58.109701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:58.109701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:58.109701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:58.109701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:58.109701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:58.109701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:58.109701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:51:58.111052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:51:58.111052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:58.111052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:58.111052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 297/595] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:58.111052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d05be-899f-4adb-a457-89eb7cda0370 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:58.179859 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb18d99e-9665-433a-b50c-01b1c32bb740 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:58.217074 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b4edb-6395-4553-8576-afc396446548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:58.218051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b4edb-6395-4553-8576-afc396446548 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:58.218473 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b4edb-6395-4553-8576-afc396446548 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:58.218988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b4edb-6395-4553-8576-afc396446548 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:58.225525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d05be-899f-4adb-a457-89eb7cda0370 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:58.228205 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b4edb-6395-4553-8576-afc396446548 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:58.229964 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b4edb-6395-4553-8576-afc396446548 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:58.244791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-734b4edb-6395-4553-8576-afc396446548 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:51:58.244791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:58.244791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:58.244791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:58.244791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:58.244791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:58.244791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:58.244791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:58.244791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:58.244791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:58.244791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:58.244791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:58.244791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:51:58.244791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:51:58.244791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:51:58.245761 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:51:58.245761 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:58.245761 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:58.245761 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:58.245761 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:58.245761 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:58.245761 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:58.245761 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:58.245761 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:58.245761 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:58.245761 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:58.245761 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:58.245761 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:58.245761 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:58.245761 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:58.245761 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:58.245761 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:58.246654 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:51:58.246654 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:51:58.246654 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:51:58.246654 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:51:58.246654 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:51:58.246654 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:51:58.246654 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:51:58.246654 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:58.254458 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 299/596] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:51:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:58.254458 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b4edb-6395-4553-8576-afc396446548 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:58.266624 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1015c8b-04ef-4420-9fdf-2ce60b108718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:58.267393 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1015c8b-04ef-4420-9fdf-2ce60b108718 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:58.267610 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1015c8b-04ef-4420-9fdf-2ce60b108718 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:58.267849 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1015c8b-04ef-4420-9fdf-2ce60b108718 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:58.279588 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1015c8b-04ef-4420-9fdf-2ce60b108718 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:58.281049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1015c8b-04ef-4420-9fdf-2ce60b108718 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:58.297739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-e1015c8b-04ef-4420-9fdf-2ce60b108718 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:58.297739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:58.297739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:58.297739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:58.297739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:58.297739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:58.297739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:58.297739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:58.297739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:58.297739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:58.297739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:58.297739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:58.297739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:51:58.297739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:51:58.297739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:51:58.297739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:51:58.298804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:58.298804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:58.298804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:58.298804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:58.298804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:58.298804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:58.298804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:58.298804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:58.298804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:58.298804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:58.298804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:58.298804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:58.298804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:58.298804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:58.298804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:58.298804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:58.298804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:51:58.299749 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:51:58.299749 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:58.299749 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:58.303664 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 298/597] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:58.304521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1015c8b-04ef-4420-9fdf-2ce60b108718 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:58.332992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:58.333551 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:58.375277 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b4edb-6395-4553-8576-afc396446548 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:58.414630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e35a1-bd4d-4c5c-aacb-33da75494484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:58.415688 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e35a1-bd4d-4c5c-aacb-33da75494484 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:58.416174 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e35a1-bd4d-4c5c-aacb-33da75494484 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:58.416623 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e35a1-bd4d-4c5c-aacb-33da75494484 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:58.417431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e35a1-bd4d-4c5c-aacb-33da75494484 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:58.418050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e35a1-bd4d-4c5c-aacb-33da75494484 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:58.419816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e35a1-bd4d-4c5c-aacb-33da75494484 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:58.425632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1015c8b-04ef-4420-9fdf-2ce60b108718 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:58.433063 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 300/598] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:58.434252 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e35a1-bd4d-4c5c-aacb-33da75494484 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:58.463415 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e339613-18d3-4bcc-a92b-e6e3128396f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:58.464060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e339613-18d3-4bcc-a92b-e6e3128396f0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:58.464341 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e339613-18d3-4bcc-a92b-e6e3128396f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:58.464615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e339613-18d3-4bcc-a92b-e6e3128396f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:58.465668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e339613-18d3-4bcc-a92b-e6e3128396f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:58.465977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e339613-18d3-4bcc-a92b-e6e3128396f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:58.468174 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e339613-18d3-4bcc-a92b-e6e3128396f0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:58.489670 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 299/599] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:51:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:58.490544 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e339613-18d3-4bcc-a92b-e6e3128396f0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:58.552646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e35a1-bd4d-4c5c-aacb-33da75494484 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:58.581026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:58.583543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:58.590367 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001814f-5af6-491c-9861-d5982ec244cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:58.591059 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001814f-5af6-491c-9861-d5982ec244cf None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:58.591343 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001814f-5af6-491c-9861-d5982ec244cf None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:58.591647 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001814f-5af6-491c-9861-d5982ec244cf None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:58.592291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b001814f-5af6-491c-9861-d5982ec244cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:58.592596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b001814f-5af6-491c-9861-d5982ec244cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:58.593973 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b001814f-5af6-491c-9861-d5982ec244cf None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:58.600768 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e339613-18d3-4bcc-a92b-e6e3128396f0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:58.607088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 301/600] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:58.608337 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34871207-dc4e-4787-a96f-7233d559151d None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:58.608626 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34871207-dc4e-4787-a96f-7233d559151d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:58.608900 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34871207-dc4e-4787-a96f-7233d559151d None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:58.614610 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 302/601] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:58.614610 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34871207-dc4e-4787-a96f-7233d559151d None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:58.645461 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e156d-af61-417e-91aa-ad85290cca22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:58.646101 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e156d-af61-417e-91aa-ad85290cca22 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:58.646382 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e156d-af61-417e-91aa-ad85290cca22 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:58.646656 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e156d-af61-417e-91aa-ad85290cca22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:58.653787 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e156d-af61-417e-91aa-ad85290cca22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:58.655227 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e156d-af61-417e-91aa-ad85290cca22 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:58.666975 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-4b5e156d-af61-417e-91aa-ad85290cca22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:51:58.666975 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:58.666975 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:58.666975 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:58.666975 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:58.666975 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:58.666975 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:58.666975 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:58.666975 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:58.666975 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:58.666975 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:58.666975 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:58.666975 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:51:58.666975 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:51:58.666975 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:51:58.667923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:51:58.667923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:58.667923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:58.667923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:58.667923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:58.667923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:58.667923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:58.667923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:58.667923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:58.667923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:58.667923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:58.667923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:58.667923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:58.667923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:58.667923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:58.667923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:58.667923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:58.668850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:51:58.668850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:51:58.668850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:51:58.668850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:51:58.668850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:51:58.668850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:51:58.668850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:51:58.668850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:58.668850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 300/602] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:51:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:58.669338 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e156d-af61-417e-91aa-ad85290cca22 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:58.725320 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34871207-dc4e-4787-a96f-7233d559151d None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:58.762889 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e31b1d-c690-4957-87b8-d37f6a16022f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:58.764005 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e31b1d-c690-4957-87b8-d37f6a16022f None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:58.764567 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e31b1d-c690-4957-87b8-d37f6a16022f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:58.765794 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e31b1d-c690-4957-87b8-d37f6a16022f None admin] PATH_INFO: `/v3/projects/fda69e8c2d5e466dbc1244b2f76296a3/users/89ff357209194276bc079d0c05898747/roles/7a33c2ceb92a478f81753573f8ece1bd` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:58.778753 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e156d-af61-417e-91aa-ad85290cca22 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:58.782430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e31b1d-c690-4957-87b8-d37f6a16022f None admin] RBAC: Authorizing `identity:create_grant(project_id=fda69e8c2d5e466dbc1244b2f76296a3, user_id=89ff357209194276bc079d0c05898747, role_id=7a33c2ceb92a478f81753573f8ece1bd)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:58.783920 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e31b1d-c690-4957-87b8-d37f6a16022f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:58.796895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 303/603] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 00:51:58 2022] PUT /identity/v3/projects/fda69e8c2d5e466dbc1244b2f76296a3/users/89ff357209194276bc079d0c05898747/roles/7a33c2ceb92a478f81753573f8ece1bd => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:51:58.797921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d7003-557c-4cb2-b465-cd4aad6ea694 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:58.798200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d7003-557c-4cb2-b465-cd4aad6ea694 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:58.798476 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d7003-557c-4cb2-b465-cd4aad6ea694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:58.821042 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-4b7d7003-557c-4cb2-b465-cd4aad6ea694 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:58.822298 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4b7d7003-557c-4cb2-b465-cd4aad6ea694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:58.830340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3aa1bb-1f00-491b-b15d-5da4aeba748a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:58.831377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3aa1bb-1f00-491b-b15d-5da4aeba748a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:58.832594 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3aa1bb-1f00-491b-b15d-5da4aeba748a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:58.833110 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3aa1bb-1f00-491b-b15d-5da4aeba748a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:58.845350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3aa1bb-1f00-491b-b15d-5da4aeba748a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:58.847935 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3aa1bb-1f00-491b-b15d-5da4aeba748a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:58.859533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:58.868628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-2a3aa1bb-1f00-491b-b15d-5da4aeba748a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:51:58.868628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:58.868628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:58.868628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:58.868628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:58.868628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:58.868628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:58.868628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:58.868628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:58.868628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:58.868628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:58.868628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:58.868628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:51:58.868628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:51:58.868628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:51:58.869613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:51:58.869613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:58.869613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:58.869613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:58.869613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:58.869613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:58.869613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:58.869613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:58.869613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:58.869613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:58.869613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:58.869613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:58.869613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:58.869613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:58.869613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:58.869613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:58.869613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:58.870506 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:51:58.870506 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:51:58.870506 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:51:58.870506 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:51:58.870506 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:51:58.870506 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:51:58.870506 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:51:58.870506 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:58.873472 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 301/604] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:51:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:58.879339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3aa1bb-1f00-491b-b15d-5da4aeba748a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:58.884441 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:58.921547 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 304/605] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:51:58.923138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d7003-557c-4cb2-b465-cd4aad6ea694 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:59.021179 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3aa1bb-1f00-491b-b15d-5da4aeba748a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:59.065967 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a4261-6319-45ad-b2db-d2752477f950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:59.067965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a4261-6319-45ad-b2db-d2752477f950 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:59.068474 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a4261-6319-45ad-b2db-d2752477f950 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:59.069116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a4261-6319-45ad-b2db-d2752477f950 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:59.070438 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a4261-6319-45ad-b2db-d2752477f950 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:59.071032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a4261-6319-45ad-b2db-d2752477f950 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:59.072604 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d7003-557c-4cb2-b465-cd4aad6ea694 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:59.072895 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a4261-6319-45ad-b2db-d2752477f950 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:59.096877 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 302/606] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:51:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:59.104227 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a4261-6319-45ad-b2db-d2752477f950 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:59.138538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54ec220-6d6e-4c4d-a450-b59bd2a742c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:59.139195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ec220-6d6e-4c4d-a450-b59bd2a742c1 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:59.139477 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ec220-6d6e-4c4d-a450-b59bd2a742c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:59.139766 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ec220-6d6e-4c4d-a450-b59bd2a742c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:59.140458 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54ec220-6d6e-4c4d-a450-b59bd2a742c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:59.141552 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54ec220-6d6e-4c4d-a450-b59bd2a742c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:59.143135 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54ec220-6d6e-4c4d-a450-b59bd2a742c1 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:59.168231 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 305/607] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:51:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:59.175701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54ec220-6d6e-4c4d-a450-b59bd2a742c1 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:59.215419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:59.228624 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:59.276887 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a4261-6319-45ad-b2db-d2752477f950 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:59.317507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b81f90-cfb7-4902-be0c-ecb15c11644d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:59.318179 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b81f90-cfb7-4902-be0c-ecb15c11644d None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:59.318469 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b81f90-cfb7-4902-be0c-ecb15c11644d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:59.318744 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b81f90-cfb7-4902-be0c-ecb15c11644d None admin] PATH_INFO: `/v3/projects/fda69e8c2d5e466dbc1244b2f76296a3/users/4d64f9bca7c546e4ba7f54c1c47d665e/roles/7a33c2ceb92a478f81753573f8ece1bd` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:59.333305 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54ec220-6d6e-4c4d-a450-b59bd2a742c1 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:59.336778 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b81f90-cfb7-4902-be0c-ecb15c11644d None admin] RBAC: Authorizing `identity:create_grant(project_id=fda69e8c2d5e466dbc1244b2f76296a3, user_id=4d64f9bca7c546e4ba7f54c1c47d665e, role_id=7a33c2ceb92a478f81753573f8ece1bd)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:59.338311 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b81f90-cfb7-4902-be0c-ecb15c11644d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:59.355510 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 303/608] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 00:51:59 2022] PUT /identity/v3/projects/fda69e8c2d5e466dbc1244b2f76296a3/users/4d64f9bca7c546e4ba7f54c1c47d665e/roles/7a33c2ceb92a478f81753573f8ece1bd => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 00:51:59.356403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b81f90-cfb7-4902-be0c-ecb15c11644d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:59.381111 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea80136-ed74-4990-8046-247487ec74aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:59.381882 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea80136-ed74-4990-8046-247487ec74aa None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:59.382237 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea80136-ed74-4990-8046-247487ec74aa None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:59.382692 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea80136-ed74-4990-8046-247487ec74aa None admin] PATH_INFO: `/v3/projects/fda69e8c2d5e466dbc1244b2f76296a3/users/9373aa297439487691fae325c0245bb9/roles/7a33c2ceb92a478f81753573f8ece1bd` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:59.402064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea80136-ed74-4990-8046-247487ec74aa None admin] RBAC: Authorizing `identity:create_grant(project_id=fda69e8c2d5e466dbc1244b2f76296a3, user_id=9373aa297439487691fae325c0245bb9, role_id=7a33c2ceb92a478f81753573f8ece1bd)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:59.403585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea80136-ed74-4990-8046-247487ec74aa None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:59.433797 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 306/609] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 00:51:59 2022] PUT /identity/v3/projects/fda69e8c2d5e466dbc1244b2f76296a3/users/9373aa297439487691fae325c0245bb9/roles/7a33c2ceb92a478f81753573f8ece1bd => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 00:51:59.491288 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b81f90-cfb7-4902-be0c-ecb15c11644d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:59.510028 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:59.535953 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a297fb4-f6f2-4341-b0bf-a94247ee687f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:59.536631 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a297fb4-f6f2-4341-b0bf-a94247ee687f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:59.536917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a297fb4-f6f2-4341-b0bf-a94247ee687f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:59.537194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a297fb4-f6f2-4341-b0bf-a94247ee687f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:59.550564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a297fb4-f6f2-4341-b0bf-a94247ee687f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:51:59.555689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-4a297fb4-f6f2-4341-b0bf-a94247ee687f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:51:59.555689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:59.555689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:59.555689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:59.555689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:59.555689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:59.555689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:59.555689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:59.555689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:59.555689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:59.555689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:59.555689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:59.555689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:51:59.555689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:51:59.555689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:51:59.555689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:51:59.556759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:51:59.556759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:51:59.556759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:51:59.556759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:51:59.556759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:59.556759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:59.556759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:59.556759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:59.556759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:59.556759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:59.556759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:59.556759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:59.556759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:59.556759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:59.556759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:59.556759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:59.557628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:59.557628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:59.557628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:59.557628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:59.557628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:51:59.557628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:51:59.557628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:51:59.557628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:51:59.557628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:51:59.557628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:51:59.557628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:51:59.557628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:51:59.557628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 304/610] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 00:51:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:59.564375 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea80136-ed74-4990-8046-247487ec74aa None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:59.652886 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e181ab-55d2-4905-a7f7-a967bf8e1adc None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:59.653189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e181ab-55d2-4905-a7f7-a967bf8e1adc None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:59.653376 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e181ab-55d2-4905-a7f7-a967bf8e1adc None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:59.655298 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 305/611] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:51:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:51:59.661745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5b7d9-3dc9-4cf8-ac4c-b6a560017139 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:59.662048 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5b7d9-3dc9-4cf8-ac4c-b6a560017139 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:59.662295 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5b7d9-3dc9-4cf8-ac4c-b6a560017139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:59.662811 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:59.696659 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea80136-ed74-4990-8046-247487ec74aa None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:59.700645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c8a5b7d9-3dc9-4cf8-ac4c-b6a560017139 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:51:59.701876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c8a5b7d9-3dc9-4cf8-ac4c-b6a560017139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:51:59.735980 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b041821-77aa-4bda-ab8e-fbe2e6ab68dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:59.736607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b041821-77aa-4bda-ab8e-fbe2e6ab68dc None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:59.736887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b041821-77aa-4bda-ab8e-fbe2e6ab68dc None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:59.737163 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b041821-77aa-4bda-ab8e-fbe2e6ab68dc None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:59.737836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b041821-77aa-4bda-ab8e-fbe2e6ab68dc None admin] RBAC: Adding query filter params (name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:51:59.738141 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b041821-77aa-4bda-ab8e-fbe2e6ab68dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:59.739537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b041821-77aa-4bda-ab8e-fbe2e6ab68dc None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:59.749499 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 307/612] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 00:51:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:51:59.758458 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b041821-77aa-4bda-ab8e-fbe2e6ab68dc None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:51:59.791527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 306/613] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:51:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:51:59.895157 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b041821-77aa-4bda-ab8e-fbe2e6ab68dc None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:51:59.943995 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44bc47-5cdf-4d4a-aca6-b0e43f5038f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:51:59.944628 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44bc47-5cdf-4d4a-aca6-b0e43f5038f0 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:51:59.944903 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44bc47-5cdf-4d4a-aca6-b0e43f5038f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:51:59.945170 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f44bc47-5cdf-4d4a-aca6-b0e43f5038f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:51:59.957341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:51:59.959793 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44bc47-5cdf-4d4a-aca6-b0e43f5038f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:51:59.961444 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f44bc47-5cdf-4d4a-aca6-b0e43f5038f0 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:51:59.984043 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-3f44bc47-5cdf-4d4a-aca6-b0e43f5038f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:59.984043 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:51:59.984043 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:51:59.984043 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:51:59.984043 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:51:59.984043 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:51:59.984043 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:51:59.984043 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:51:59.984043 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:51:59.984043 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:51:59.984043 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:51:59.984043 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:51:59.984043 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:51:59.984043 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:51:59.984043 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:51:59.984043 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:51:59.986572 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:51:59.986572 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:51:59.986572 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:51:59.986572 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:51:59.986572 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:51:59.986572 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:51:59.986572 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:51:59.986572 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:51:59.986572 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:51:59.986572 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:51:59.986572 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:51:59.986572 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:51:59.986572 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:51:59.986572 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:51:59.986572 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:51:59.986572 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:51:59.986572 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:51:59.988536 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:51:59.988536 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:51:59.988536 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:51:59.988536 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 308/614] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:51:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:51:59.992829 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5b7d9-3dc9-4cf8-ac4c-b6a560017139 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:00.105520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5b7d9-3dc9-4cf8-ac4c-b6a560017139 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:00.107918 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:00.154992 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0a451-2bf5-42ee-a387-1d20b8aaae51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:00.155629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0a451-2bf5-42ee-a387-1d20b8aaae51 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:00.155922 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0a451-2bf5-42ee-a387-1d20b8aaae51 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:00.156202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0a451-2bf5-42ee-a387-1d20b8aaae51 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:00.156954 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d0a451-2bf5-42ee-a387-1d20b8aaae51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:00.157261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d0a451-2bf5-42ee-a387-1d20b8aaae51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:00.158734 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d0a451-2bf5-42ee-a387-1d20b8aaae51 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:00.176415 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 307/615] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:00.181675 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44bc47-5cdf-4d4a-aca6-b0e43f5038f0 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:00.204177 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0a451-2bf5-42ee-a387-1d20b8aaae51 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:00.331275 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0a451-2bf5-42ee-a387-1d20b8aaae51 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:00.367408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e619fe6-b9d9-4f23-bd39-d3fbbb0181d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:00.368181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e619fe6-b9d9-4f23-bd39-d3fbbb0181d4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:00.368459 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e619fe6-b9d9-4f23-bd39-d3fbbb0181d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:00.368832 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e619fe6-b9d9-4f23-bd39-d3fbbb0181d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:00.379535 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e619fe6-b9d9-4f23-bd39-d3fbbb0181d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:52:00.384826 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f44bc47-5cdf-4d4a-aca6-b0e43f5038f0 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:00.385036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-4e619fe6-b9d9-4f23-bd39-d3fbbb0181d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:52:00.385036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:00.385036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:00.385036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:00.385036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:00.385036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:00.385036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:00.385036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:00.385036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:00.385036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:00.385036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:00.385036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:00.385036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:52:00.385036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:52:00.385036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:52:00.385036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:52:00.387083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:52:00.387083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:52:00.387083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:52:00.387083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:52:00.387083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:00.387083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:00.387083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:00.387083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:00.387083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:00.387083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:00.387083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:00.387083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:00.387083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:00.387083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:00.387083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:00.387083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:00.388266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:00.388266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:00.388266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:00.388266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:00.388266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:52:00.388266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:52:00.388266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:52:00.388266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:52:00.388266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:52:00.388266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:52:00.388266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:52:00.388266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:00.388266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 308/616] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 00:52:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:52:00.392604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e619fe6-b9d9-4f23-bd39-d3fbbb0181d4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:00.442765 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:00.454142 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22394784-dd98-4889-bec4-aefe34a26c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:00.455101 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22394784-dd98-4889-bec4-aefe34a26c26 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:00.457563 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22394784-dd98-4889-bec4-aefe34a26c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:00.458024 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22394784-dd98-4889-bec4-aefe34a26c26 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:00.473213 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22394784-dd98-4889-bec4-aefe34a26c26 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:00.476444 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22394784-dd98-4889-bec4-aefe34a26c26 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:00.496518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-22394784-dd98-4889-bec4-aefe34a26c26 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 00:52:00.496518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:00.496518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:00.496518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:00.496518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:00.496518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:00.496518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:00.496518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:00.496518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:00.496518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:00.496518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:00.496518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:00.496518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:52:00.496518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:52:00.496518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:52:00.496518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:52:00.497770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:00.497770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:00.497770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:52:00.497770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:00.497770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:52:00.497770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:00.497770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:00.497770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:00.497770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:00.497770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:00.497770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:00.497770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:00.497770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:00.497770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:00.497770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:00.497770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:00.497770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:00.497770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:00.498970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:00.498970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:00.498970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:52:00.498970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:52:00.498970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:52:00.498970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:52:00.498970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:52:00.498970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:52:00.498970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 00:52:00.498970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:00.507000 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 309/617] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Sep 14 00:52:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:00.511296 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22394784-dd98-4889-bec4-aefe34a26c26 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:00.591042 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e619fe6-b9d9-4f23-bd39-d3fbbb0181d4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:00.611513 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:00.659496 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aaae0a-3bd1-4bef-89f6-a02c4e68e7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:00.660137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aaae0a-3bd1-4bef-89f6-a02c4e68e7dd None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:00.660415 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aaae0a-3bd1-4bef-89f6-a02c4e68e7dd None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:00.660689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aaae0a-3bd1-4bef-89f6-a02c4e68e7dd None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:00.661347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aaae0a-3bd1-4bef-89f6-a02c4e68e7dd None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:00.661647 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aaae0a-3bd1-4bef-89f6-a02c4e68e7dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:00.663009 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aaae0a-3bd1-4bef-89f6-a02c4e68e7dd None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:00.671531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 309/618] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 00:52:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:00.679206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aaae0a-3bd1-4bef-89f6-a02c4e68e7dd None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:00.684580 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22394784-dd98-4889-bec4-aefe34a26c26 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:00.723384 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824feca9-817a-44db-a6cc-640d6172ace0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:00.723959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824feca9-817a-44db-a6cc-640d6172ace0 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:00.724240 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824feca9-817a-44db-a6cc-640d6172ace0 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:00.724516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824feca9-817a-44db-a6cc-640d6172ace0 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:00.725255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824feca9-817a-44db-a6cc-640d6172ace0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:00.725562 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824feca9-817a-44db-a6cc-640d6172ace0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:00.727016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824feca9-817a-44db-a6cc-640d6172ace0 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:00.747509 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 310/619] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Sep 14 00:52:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:00.756346 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824feca9-817a-44db-a6cc-640d6172ace0 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:00.847309 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:00.875355 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aaae0a-3bd1-4bef-89f6-a02c4e68e7dd None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:00.932234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824feca9-817a-44db-a6cc-640d6172ace0 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:00.944289 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dab0b2-e7bc-45d4-b1b7-602c9cd8066d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:00.944481 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:00.945157 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dab0b2-e7bc-45d4-b1b7-602c9cd8066d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:00.945543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dab0b2-e7bc-45d4-b1b7-602c9cd8066d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:00.945918 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dab0b2-e7bc-45d4-b1b7-602c9cd8066d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:00.970571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dab0b2-e7bc-45d4-b1b7-602c9cd8066d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:00.972373 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dab0b2-e7bc-45d4-b1b7-602c9cd8066d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:00.981169 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69cae7-3c33-4034-9b30-d6f9a659b603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:00.982769 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69cae7-3c33-4034-9b30-d6f9a659b603 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:00.983202 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69cae7-3c33-4034-9b30-d6f9a659b603 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:00.983836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69cae7-3c33-4034-9b30-d6f9a659b603 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:00.996283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-27dab0b2-e7bc-45d4-b1b7-602c9cd8066d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:00.996283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:00.996283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:00.996283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:00.996283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:00.996283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:00.996283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:00.996283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:00.996283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:00.996283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:00.996283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:00.996283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:00.996283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:00.996283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:00.996283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:00.996283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:00.997494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:00.997494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:00.997494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:00.997494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:00.997494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:00.997494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:00.997494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:00.997494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:00.997494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:00.997494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:00.997494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:00.997494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:00.997494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:00.997494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:00.997494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:00.997494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:00.997494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:00.998597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:00.998597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:00.998597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:01.005125 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 310/620] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:01.013762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dab0b2-e7bc-45d4-b1b7-602c9cd8066d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:01.139974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba69cae7-3c33-4034-9b30-d6f9a659b603 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:01.142673 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba69cae7-3c33-4034-9b30-d6f9a659b603 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:01.150479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dab0b2-e7bc-45d4-b1b7-602c9cd8066d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:01.196675 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d5e40-97b5-4b0d-b7c3-e2e23bf03f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:01.197551 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d5e40-97b5-4b0d-b7c3-e2e23bf03f32 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:01.197985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d5e40-97b5-4b0d-b7c3-e2e23bf03f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:01.198420 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d5e40-97b5-4b0d-b7c3-e2e23bf03f32 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:01.199206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d5e40-97b5-4b0d-b7c3-e2e23bf03f32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:01.199800 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d5e40-97b5-4b0d-b7c3-e2e23bf03f32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:01.201705 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d5e40-97b5-4b0d-b7c3-e2e23bf03f32 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:01.214632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:01.225028 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 311/621] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:01.230549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d5e40-97b5-4b0d-b7c3-e2e23bf03f32 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:01.326635 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 311/622] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:52:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:52:01.333302 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69cae7-3c33-4034-9b30-d6f9a659b603 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:01.354312 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:01.429710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d5e40-97b5-4b0d-b7c3-e2e23bf03f32 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:01.479093 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53471de-b261-4583-9aa9-1c4efd5dbdcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:01.479446 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53471de-b261-4583-9aa9-1c4efd5dbdcc None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:01.479654 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53471de-b261-4583-9aa9-1c4efd5dbdcc None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:01.479939 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53471de-b261-4583-9aa9-1c4efd5dbdcc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:01.496376 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53471de-b261-4583-9aa9-1c4efd5dbdcc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:01.497846 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53471de-b261-4583-9aa9-1c4efd5dbdcc None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:01.522060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-d53471de-b261-4583-9aa9-1c4efd5dbdcc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 00:52:01.522060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:01.522060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:01.522060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:01.522060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:01.522060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:01.522060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:01.522060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:01.522060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:01.522060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:01.522060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:01.522060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:01.522060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:52:01.522060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:52:01.522060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:52:01.522060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:52:01.523062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:01.523062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:01.523062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:52:01.523062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:01.523062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:52:01.523062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:01.523062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:01.523062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:01.523062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:01.523062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:01.523062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:01.523062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:01.523062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:01.523062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:01.523062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:01.523062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:01.523062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:01.523062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:01.524057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:01.524057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:01.524057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:52:01.524057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:52:01.524057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:52:01.524057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:52:01.524057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:52:01.524057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:52:01.524057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 00:52:01.524057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:01.529493 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 312/623] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:52:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:52:01.530366 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53471de-b261-4583-9aa9-1c4efd5dbdcc None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:01.549832 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69cae7-3c33-4034-9b30-d6f9a659b603 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:01.596972 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c5e3a-1960-46ab-9f4f-b053cbe56f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:01.597592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c5e3a-1960-46ab-9f4f-b053cbe56f34 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:01.597876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c5e3a-1960-46ab-9f4f-b053cbe56f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:01.598154 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c5e3a-1960-46ab-9f4f-b053cbe56f34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:01.609069 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c5e3a-1960-46ab-9f4f-b053cbe56f34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:01.611210 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c5e3a-1960-46ab-9f4f-b053cbe56f34 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:01.620995 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:01.627853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-7c6c5e3a-1960-46ab-9f4f-b053cbe56f34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:01.627853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:01.627853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:01.627853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:01.627853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:01.627853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:01.627853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:01.627853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:01.627853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:01.627853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:01.627853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:01.627853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:01.627853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:01.627853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:01.627853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:01.627853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:01.628843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:01.628843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:01.628843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:01.628843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:01.628843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:01.628843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:01.628843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:01.628843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:01.628843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:01.628843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:01.628843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:01.628843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:01.628843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:01.628843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:01.628843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:01.628843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:01.628843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:01.629753 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:01.629753 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:01.629753 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:01.633607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 312/624] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:01.634854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cdd6f-c7dc-46ec-98db-c2cdd744ffd5 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:01.635118 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cdd6f-c7dc-46ec-98db-c2cdd744ffd5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:01.635410 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cdd6f-c7dc-46ec-98db-c2cdd744ffd5 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:01.641629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 313/625] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:01.641629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54cdd6f-c7dc-46ec-98db-c2cdd744ffd5 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:01.676109 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53471de-b261-4583-9aa9-1c4efd5dbdcc None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:01.712525 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:01.714850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea164e-e0dd-4222-a803-140748b7d531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:01.715924 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea164e-e0dd-4222-a803-140748b7d531 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:01.717266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea164e-e0dd-4222-a803-140748b7d531 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:01.717975 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea164e-e0dd-4222-a803-140748b7d531 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:01.718935 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea164e-e0dd-4222-a803-140748b7d531 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:01.719991 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea164e-e0dd-4222-a803-140748b7d531 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:01.721746 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea164e-e0dd-4222-a803-140748b7d531 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:01.741291 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 313/626] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Sep 14 00:52:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:01.743097 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a4266-750f-4026-a612-13c95d86f6c2 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:01.743651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a4266-750f-4026-a612-13c95d86f6c2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:01.744124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a4266-750f-4026-a612-13c95d86f6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:01.756849 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54cdd6f-c7dc-46ec-98db-c2cdd744ffd5 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:01.764248 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-363a4266-750f-4026-a612-13c95d86f6c2 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:01.766188 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-363a4266-750f-4026-a612-13c95d86f6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:01.799331 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ce730-3cf1-4bd3-81c0-a7f5f9e01163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:01.800663 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ce730-3cf1-4bd3-81c0-a7f5f9e01163 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:01.801149 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ce730-3cf1-4bd3-81c0-a7f5f9e01163 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:01.802138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ce730-3cf1-4bd3-81c0-a7f5f9e01163 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:01.803476 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ce730-3cf1-4bd3-81c0-a7f5f9e01163 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:01.804340 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ce730-3cf1-4bd3-81c0-a7f5f9e01163 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:01.806827 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ce730-3cf1-4bd3-81c0-a7f5f9e01163 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:01.823822 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 314/627] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:01.825038 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ce730-3cf1-4bd3-81c0-a7f5f9e01163 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:01.857065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 314/628] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:52:01.858218 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a4266-750f-4026-a612-13c95d86f6c2 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:01.917759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:01.941554 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ce730-3cf1-4bd3-81c0-a7f5f9e01163 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:01.967245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:01.969940 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a4266-750f-4026-a612-13c95d86f6c2 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:01.981986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621671c-8833-44cf-8512-f454c7a96c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:01.982893 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621671c-8833-44cf-8512-f454c7a96c4c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:01.983396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621671c-8833-44cf-8512-f454c7a96c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:01.983888 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621671c-8833-44cf-8512-f454c7a96c4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:01.995174 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6621671c-8833-44cf-8512-f454c7a96c4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:01.997116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6621671c-8833-44cf-8512-f454c7a96c4c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:02.007797 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70459346-ad4e-484b-8f12-d15c8525985e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:02.008457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70459346-ad4e-484b-8f12-d15c8525985e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:02.008742 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70459346-ad4e-484b-8f12-d15c8525985e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:02.009017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70459346-ad4e-484b-8f12-d15c8525985e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:02.018249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-6621671c-8833-44cf-8512-f454c7a96c4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:02.018249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:02.018249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:02.018249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:02.018249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:02.018249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:02.018249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:02.018249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:02.018249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:02.018249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:02.018249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:02.018249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:02.018249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:02.018249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:02.018249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:02.018249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:02.019310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:02.019310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:02.019310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:02.019310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:02.019310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:02.019310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:02.019310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:02.019310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:02.019310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:02.019310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:02.019310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:02.019310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:02.019310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:02.019310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:02.019310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:02.019310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:02.019310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:02.020239 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:02.020239 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:02.020239 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:02.020239 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 315/629] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:02.021869 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70459346-ad4e-484b-8f12-d15c8525985e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:02.023367 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70459346-ad4e-484b-8f12-d15c8525985e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:02.024100 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da02e7e-7c1d-4505-9d0b-0b766bd26245 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:02.024584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da02e7e-7c1d-4505-9d0b-0b766bd26245 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:02.025079 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da02e7e-7c1d-4505-9d0b-0b766bd26245 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:02.027922 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 316/630] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:02.029104 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da02e7e-7c1d-4505-9d0b-0b766bd26245 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:02.039451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-70459346-ad4e-484b-8f12-d15c8525985e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:02.039451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:02.039451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:02.039451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:02.039451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:02.039451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:02.039451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:02.039451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:02.039451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:02.039451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:02.039451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:02.039451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:02.039451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:52:02.039451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:52:02.039451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:52:02.040498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:52:02.040498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:02.040498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:02.040498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:02.040498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:02.040498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:02.040498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:02.040498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:02.040498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:02.040498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:02.040498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:02.040498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:02.040498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:02.040498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:02.040498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:02.040498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:02.040498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:02.041401 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:52:02.041401 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:52:02.041401 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:52:02.041401 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:52:02.041401 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:52:02.041401 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:52:02.041401 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:02.041401 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:02.043649 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 315/631] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:02.045313 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4042c98-389f-4a1e-ad0c-3de4a50fd343 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:02.045874 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4042c98-389f-4a1e-ad0c-3de4a50fd343 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:02.048083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4042c98-389f-4a1e-ad0c-3de4a50fd343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:02.069080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c4042c98-389f-4a1e-ad0c-3de4a50fd343 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:02.070882 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c4042c98-389f-4a1e-ad0c-3de4a50fd343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:02.155660 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 316/632] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:52:02.155660 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4042c98-389f-4a1e-ad0c-3de4a50fd343 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:02.169558 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da02e7e-7c1d-4505-9d0b-0b766bd26245 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:02.193548 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:02.206322 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddbd833-714d-4548-8807-eec63acbd6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:02.206964 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbd833-714d-4548-8807-eec63acbd6ad None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:02.207252 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbd833-714d-4548-8807-eec63acbd6ad None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:02.207527 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbd833-714d-4548-8807-eec63acbd6ad None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:02.208199 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddbd833-714d-4548-8807-eec63acbd6ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:02.208506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddbd833-714d-4548-8807-eec63acbd6ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:02.209882 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddbd833-714d-4548-8807-eec63acbd6ad None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:02.226508 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 317/633] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:02.226508 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75442ec0-5924-4c58-b9a9-e2cb3546d663 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:02.226508 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75442ec0-5924-4c58-b9a9-e2cb3546d663 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:02.226508 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75442ec0-5924-4c58-b9a9-e2cb3546d663 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:02.226508 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 318/634] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:02.226508 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75442ec0-5924-4c58-b9a9-e2cb3546d663 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:02.262549 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:02.268908 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4042c98-389f-4a1e-ad0c-3de4a50fd343 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:02.306744 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebeb95-11ce-48fd-aadd-a7d4143c0f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:02.307358 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebeb95-11ce-48fd-aadd-a7d4143c0f80 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:02.307679 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebeb95-11ce-48fd-aadd-a7d4143c0f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:02.307956 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebeb95-11ce-48fd-aadd-a7d4143c0f80 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:02.308652 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebeb95-11ce-48fd-aadd-a7d4143c0f80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:02.308985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebeb95-11ce-48fd-aadd-a7d4143c0f80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:02.310393 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebeb95-11ce-48fd-aadd-a7d4143c0f80 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:02.321830 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 317/635] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:52:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:02.322642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebeb95-11ce-48fd-aadd-a7d4143c0f80 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:02.341283 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75442ec0-5924-4c58-b9a9-e2cb3546d663 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:02.378312 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6a61e-ed59-4e15-88a6-c6ad8ce0eceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:02.379061 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6a61e-ed59-4e15-88a6-c6ad8ce0eceb None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:02.379344 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6a61e-ed59-4e15-88a6-c6ad8ce0eceb None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:02.379651 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6a61e-ed59-4e15-88a6-c6ad8ce0eceb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:02.390175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6a61e-ed59-4e15-88a6-c6ad8ce0eceb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:52:02.396140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-c6c6a61e-ed59-4e15-88a6-c6ad8ce0eceb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:52:02.396140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:02.396140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:02.396140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:02.396140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:02.396140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:02.396140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:02.396140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:02.396140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:02.396140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:02.396140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:02.396140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:02.396140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:52:02.396140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:52:02.396140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:52:02.396140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:52:02.397071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:52:02.397071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:52:02.397071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:52:02.397071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:52:02.397071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:02.397071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:02.397071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:02.397071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:02.397071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:02.397071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:02.397071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:02.397071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:02.397071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:02.397071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:02.397071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:02.397071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:02.397933 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:02.397933 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:02.397933 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:02.397933 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:02.397933 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:52:02.397933 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:52:02.397933 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:52:02.397933 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:52:02.397933 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:52:02.397933 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:52:02.397933 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:52:02.397933 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:02.397933 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 319/636] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 00:52:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:02.398650 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31218d8-b064-4ea5-b31f-c7cbf55f2eb1 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:02.399205 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31218d8-b064-4ea5-b31f-c7cbf55f2eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:02.399205 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31218d8-b064-4ea5-b31f-c7cbf55f2eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:02.417758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d31218d8-b064-4ea5-b31f-c7cbf55f2eb1 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:02.418949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d31218d8-b064-4ea5-b31f-c7cbf55f2eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:02.427311 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:02.433707 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebeb95-11ce-48fd-aadd-a7d4143c0f80 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:02.467899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83cdca-3f49-4549-a72e-4fd128ed7d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:02.468529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83cdca-3f49-4549-a72e-4fd128ed7d1c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:02.468813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83cdca-3f49-4549-a72e-4fd128ed7d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:02.469123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83cdca-3f49-4549-a72e-4fd128ed7d1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:02.475943 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83cdca-3f49-4549-a72e-4fd128ed7d1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:02.477479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83cdca-3f49-4549-a72e-4fd128ed7d1c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:02.489044 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-6e83cdca-3f49-4549-a72e-4fd128ed7d1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:02.489044 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:02.489044 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:02.489044 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:02.489044 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:02.489044 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:02.489044 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:02.489044 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:02.489044 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:02.489044 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:02.489044 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:02.489044 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:02.489044 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:52:02.489044 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:52:02.489044 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:52:02.490134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:52:02.490134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:02.490134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:02.490134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:02.490134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:02.490134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:02.490134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:02.490134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:02.490134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:02.490134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:02.490134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:02.490134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:02.490134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:02.490134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:02.490134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:02.490134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:02.490134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:02.491261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:52:02.491261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:52:02.491261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:52:02.491261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:52:02.491261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:52:02.491261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:52:02.491261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:02.491261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:02.491261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 318/637] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:02.491261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83cdca-3f49-4549-a72e-4fd128ed7d1c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:02.495772 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 320/638] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:52:02.495772 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31218d8-b064-4ea5-b31f-c7cbf55f2eb1 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:02.532340 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:02.602082 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31218d8-b064-4ea5-b31f-c7cbf55f2eb1 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:02.603417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83cdca-3f49-4549-a72e-4fd128ed7d1c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:02.639152 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e138e-41ec-4794-9c5a-bf9cbd53125e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:02.639787 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e138e-41ec-4794-9c5a-bf9cbd53125e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:02.640068 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e138e-41ec-4794-9c5a-bf9cbd53125e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:02.640336 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e138e-41ec-4794-9c5a-bf9cbd53125e None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:02.642005 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e138e-41ec-4794-9c5a-bf9cbd53125e None admin] RBAC: Adding query filter params (name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:02.642427 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e138e-41ec-4794-9c5a-bf9cbd53125e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:02.642532 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9c0c3-d910-4746-9a40-8e5e7026d964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:02.643179 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9c0c3-d910-4746-9a40-8e5e7026d964 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:02.643528 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9c0c3-d910-4746-9a40-8e5e7026d964 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:02.643847 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e138e-41ec-4794-9c5a-bf9cbd53125e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:02.643944 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9c0c3-d910-4746-9a40-8e5e7026d964 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:02.644719 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9c0c3-d910-4746-9a40-8e5e7026d964 None admin] RBAC: Adding query filter params (role.id=7a33c2ceb92a478f81753573f8ece1bd, scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3, user.id=5e0fe2e84f6748c992c9beedf2f5d49b) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:02.645096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9c0c3-d910-4746-9a40-8e5e7026d964 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:02.646511 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9c0c3-d910-4746-9a40-8e5e7026d964 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:02.659872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 321/639] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 00:52:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:02.660032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 319/640] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:52:02 2022] GET /identity/v3/role_assignments?user.id=5e0fe2e84f6748c992c9beedf2f5d49b&scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3&role.id=7a33c2ceb92a478f81753573f8ece1bd => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:02.660112 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e138e-41ec-4794-9c5a-bf9cbd53125e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:02.660185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9c0c3-d910-4746-9a40-8e5e7026d964 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:02.683675 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:02.762604 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:02.780689 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e138e-41ec-4794-9c5a-bf9cbd53125e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:02.799662 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9c0c3-d910-4746-9a40-8e5e7026d964 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:02.817225 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f81bcf-5d58-4717-ac12-fff94ec0f583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:02.817876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f81bcf-5d58-4717-ac12-fff94ec0f583 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:02.818162 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f81bcf-5d58-4717-ac12-fff94ec0f583 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:02.818437 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f81bcf-5d58-4717-ac12-fff94ec0f583 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:02.832293 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f81bcf-5d58-4717-ac12-fff94ec0f583 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:52:02.837723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-86f81bcf-5d58-4717-ac12-fff94ec0f583 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:52:02.837723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:02.837723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:02.837723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:02.837723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:02.837723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:02.837723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:02.837723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:02.837723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:02.837723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:02.837723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:02.837723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:02.837723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:52:02.837723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:52:02.837723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:52:02.837723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:52:02.838704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:52:02.838704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:52:02.838704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:52:02.838704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:52:02.838704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:02.838704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:02.838704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:02.838704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:02.838704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:02.838704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:02.838704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:02.838704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:02.838704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:02.838704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:02.838704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:02.838704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:02.839579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:02.839579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:02.839579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:02.839579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:02.839579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:52:02.839579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:52:02.839579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:52:02.839579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:52:02.839579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:52:02.839579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:52:02.839579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:52:02.839579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:02.839579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 322/641] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 00:52:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:02.840299 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f81bcf-5d58-4717-ac12-fff94ec0f583 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:02.847191 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5704d60-0a85-401c-b052-9d8c202c4a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:02.848582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5704d60-0a85-401c-b052-9d8c202c4a44 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:02.848582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5704d60-0a85-401c-b052-9d8c202c4a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:02.848582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5704d60-0a85-401c-b052-9d8c202c4a44 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:02.849131 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5704d60-0a85-401c-b052-9d8c202c4a44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:02.849351 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5704d60-0a85-401c-b052-9d8c202c4a44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:02.851064 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5704d60-0a85-401c-b052-9d8c202c4a44 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:02.866035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 320/642] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:52:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:02.866892 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5704d60-0a85-401c-b052-9d8c202c4a44 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:02.955381 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f81bcf-5d58-4717-ac12-fff94ec0f583 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:02.962197 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:02.985974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5704d60-0a85-401c-b052-9d8c202c4a44 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:02.991439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05e27b-ce0b-421a-b365-5dc463dcea8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:02.992106 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05e27b-ce0b-421a-b365-5dc463dcea8a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:02.992382 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05e27b-ce0b-421a-b365-5dc463dcea8a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:02.992650 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05e27b-ce0b-421a-b365-5dc463dcea8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:03.006545 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce05e27b-ce0b-421a-b365-5dc463dcea8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:03.008038 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce05e27b-ce0b-421a-b365-5dc463dcea8a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:03.016506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:03.023705 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f817061-3db2-4a29-829b-7f3567fc01f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:03.024345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f817061-3db2-4a29-829b-7f3567fc01f0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:03.024623 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f817061-3db2-4a29-829b-7f3567fc01f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:03.024889 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f817061-3db2-4a29-829b-7f3567fc01f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:03.027201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-ce05e27b-ce0b-421a-b365-5dc463dcea8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:03.027201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:03.027201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:03.027201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:03.027201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:03.027201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:03.027201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:03.027201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:03.027201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:03.027201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:03.027201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:03.027201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:03.027201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:03.027201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:03.027201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:03.027201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:03.028247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:03.028247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:03.028247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:03.028247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:03.028247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:03.028247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:03.028247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:03.028247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:03.028247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:03.028247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:03.028247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:03.028247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:03.028247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:03.028247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:03.028247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:03.028247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:03.028247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:03.029183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:03.029183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:03.029183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:03.032379 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 323/643] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:03.034623 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05e27b-ce0b-421a-b365-5dc463dcea8a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:03.042795 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f817061-3db2-4a29-829b-7f3567fc01f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:52:03.047731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-2f817061-3db2-4a29-829b-7f3567fc01f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:52:03.047731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:03.047731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:03.047731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:03.047731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:03.047731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:03.047731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:03.047731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:03.047731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:03.047731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:03.047731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:03.047731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:03.047731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:52:03.047731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:52:03.047731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:52:03.047731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:52:03.048655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:52:03.048655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:52:03.048655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:52:03.048655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:52:03.048655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:03.048655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:03.048655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:03.048655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:03.048655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:03.048655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:03.048655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:03.048655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:03.048655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:03.048655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:03.048655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:03.048655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:03.049639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:03.049639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:03.049639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:03.049639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:03.049639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:52:03.049639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:52:03.049639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:52:03.049639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:52:03.049639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:52:03.049639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:52:03.049639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:52:03.049639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:03.049639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 321/644] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 00:52:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:03.049639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f817061-3db2-4a29-829b-7f3567fc01f0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:03.152524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05e27b-ce0b-421a-b365-5dc463dcea8a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:03.160888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f817061-3db2-4a29-829b-7f3567fc01f0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:03.186724 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ad0e5-230b-4c5b-bb13-bf4073107585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:03.187340 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ad0e5-230b-4c5b-bb13-bf4073107585 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:03.187643 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ad0e5-230b-4c5b-bb13-bf4073107585 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:03.188510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ad0e5-230b-4c5b-bb13-bf4073107585 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:03.189278 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842ad0e5-230b-4c5b-bb13-bf4073107585 None admin] RBAC: Adding query filter params (name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:03.189582 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842ad0e5-230b-4c5b-bb13-bf4073107585 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:03.191076 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842ad0e5-230b-4c5b-bb13-bf4073107585 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:03.197803 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2ee29-73b2-42a2-a47b-6fa9e6b212bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:03.198461 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2ee29-73b2-42a2-a47b-6fa9e6b212bc None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:03.198738 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2ee29-73b2-42a2-a47b-6fa9e6b212bc None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:03.199008 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2ee29-73b2-42a2-a47b-6fa9e6b212bc None admin] PATH_INFO: `/v3/projects/fda69e8c2d5e466dbc1244b2f76296a3/users/e52776d9b4894af9b85a24136a8255e5/roles/7a33c2ceb92a478f81753573f8ece1bd` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:03.199612 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 324/645] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 00:52:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:03.200421 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ad0e5-230b-4c5b-bb13-bf4073107585 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:03.215740 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:03.220325 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2ee29-73b2-42a2-a47b-6fa9e6b212bc None admin] RBAC: Authorizing `identity:create_grant(project_id=fda69e8c2d5e466dbc1244b2f76296a3, user_id=e52776d9b4894af9b85a24136a8255e5, role_id=7a33c2ceb92a478f81753573f8ece1bd)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:03.222372 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2ee29-73b2-42a2-a47b-6fa9e6b212bc None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:03.236310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 322/646] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 00:52:03 2022] PUT /identity/v3/projects/fda69e8c2d5e466dbc1244b2f76296a3/users/e52776d9b4894af9b85a24136a8255e5/roles/7a33c2ceb92a478f81753573f8ece1bd => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:52:03.237141 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2ee29-73b2-42a2-a47b-6fa9e6b212bc None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:03.286881 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:03.340385 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ad0e5-230b-4c5b-bb13-bf4073107585 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:03.380030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d4d44-8957-4004-a764-f677b37d14f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:03.380244 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2ee29-73b2-42a2-a47b-6fa9e6b212bc None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:03.381168 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d4d44-8957-4004-a764-f677b37d14f6 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:03.381758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d4d44-8957-4004-a764-f677b37d14f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:03.382255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d4d44-8957-4004-a764-f677b37d14f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:03.383267 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605d4d44-8957-4004-a764-f677b37d14f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:03.383870 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605d4d44-8957-4004-a764-f677b37d14f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:03.385578 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605d4d44-8957-4004-a764-f677b37d14f6 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:03.410242 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 325/647] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:03.410242 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d4d44-8957-4004-a764-f677b37d14f6 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:03.435115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee1eb4-7b92-48ba-a51e-113ed8e8426e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:03.435805 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee1eb4-7b92-48ba-a51e-113ed8e8426e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:03.436090 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee1eb4-7b92-48ba-a51e-113ed8e8426e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:03.436364 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee1eb4-7b92-48ba-a51e-113ed8e8426e None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:03.437512 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee1eb4-7b92-48ba-a51e-113ed8e8426e None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:03.438395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee1eb4-7b92-48ba-a51e-113ed8e8426e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:03.439890 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee1eb4-7b92-48ba-a51e-113ed8e8426e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:03.449028 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 323/648] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 00:52:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:03.449709 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee1eb4-7b92-48ba-a51e-113ed8e8426e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:03.518194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:03.536481 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d4d44-8957-4004-a764-f677b37d14f6 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:03.559034 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:03.575231 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1855b9-33ad-472c-8da9-dce4d1300cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:03.576116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1855b9-33ad-472c-8da9-dce4d1300cce None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:03.576539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1855b9-33ad-472c-8da9-dce4d1300cce None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:03.576837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee1eb4-7b92-48ba-a51e-113ed8e8426e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:03.577118 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1855b9-33ad-472c-8da9-dce4d1300cce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:03.588589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1855b9-33ad-472c-8da9-dce4d1300cce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:03.590231 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1855b9-33ad-472c-8da9-dce4d1300cce None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:03.605287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-0e1855b9-33ad-472c-8da9-dce4d1300cce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:03.605287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:03.605287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:03.605287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:03.605287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:03.605287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:03.605287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:03.605287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:03.605287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:03.605287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:03.605287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:03.605287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:03.605287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:03.605287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:03.605287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:03.605287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:03.606463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:03.606463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:03.606463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:03.606463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:03.606463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:03.606463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:03.606463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:03.606463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:03.606463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:03.606463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:03.606463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:03.606463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:03.606463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:03.606463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:03.606463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:03.606463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:03.606463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:03.607451 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:03.607451 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:03.607451 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:03.609177 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 326/649] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:03.610800 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1855b9-33ad-472c-8da9-dce4d1300cce None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:03.616704 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554d80c-8d4a-4feb-90c4-f0be1dd5e0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:03.617349 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554d80c-8d4a-4feb-90c4-f0be1dd5e0f2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:03.617629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554d80c-8d4a-4feb-90c4-f0be1dd5e0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:03.617904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554d80c-8d4a-4feb-90c4-f0be1dd5e0f2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:03.626378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4554d80c-8d4a-4feb-90c4-f0be1dd5e0f2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:03.627875 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4554d80c-8d4a-4feb-90c4-f0be1dd5e0f2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:03.642322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-4554d80c-8d4a-4feb-90c4-f0be1dd5e0f2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 14 00:52:03.642322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:03.642322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:03.642322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:03.642322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:03.642322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:03.642322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:03.642322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:03.642322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:03.642322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:03.642322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:03.642322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:03.642322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:52:03.642322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:52:03.642322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:52:03.642322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:52:03.643416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:03.643416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:03.643416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:52:03.643416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:03.643416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:52:03.643416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:03.643416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:03.643416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:03.643416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:03.643416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:03.643416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:03.643416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:03.643416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:03.643416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:03.643416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:03.643416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:03.643416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:03.643416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:03.644872 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:03.644872 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:03.644872 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:52:03.644872 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:52:03.644872 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:52:03.644872 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:52:03.644872 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:52:03.644872 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:52:03.644872 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 14 00:52:03.644872 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:03.644872 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 324/650] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:52:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:03.645499 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554d80c-8d4a-4feb-90c4-f0be1dd5e0f2 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:03.727280 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1855b9-33ad-472c-8da9-dce4d1300cce None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:03.772504 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba9aba-711d-4a03-bae3-4f1de72f7498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:03.773089 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba9aba-711d-4a03-bae3-4f1de72f7498 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:03.773371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba9aba-711d-4a03-bae3-4f1de72f7498 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:03.773646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba9aba-711d-4a03-bae3-4f1de72f7498 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:03.773966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554d80c-8d4a-4feb-90c4-f0be1dd5e0f2 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:03.788041 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba9aba-711d-4a03-bae3-4f1de72f7498 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:03.788980 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:03.791722 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba9aba-711d-4a03-bae3-4f1de72f7498 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:03.811539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-15ba9aba-711d-4a03-bae3-4f1de72f7498 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:03.811539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:03.811539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:03.811539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:03.811539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:03.811539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:03.811539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:03.811539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:03.811539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:03.811539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:03.811539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:03.811539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:03.811539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:03.811539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:03.811539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:03.811539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:03.812541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:03.812541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:03.812541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:03.812541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:03.812541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:03.812541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:03.812541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:03.812541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:03.812541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:03.812541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:03.812541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:03.812541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:03.812541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:03.812541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:03.812541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:03.812541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:03.812541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:03.813514 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:03.813514 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:03.813514 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:03.813514 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 327/651] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:03.814333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba9aba-711d-4a03-bae3-4f1de72f7498 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:03.819112 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da6c41-7ca2-4861-990d-585377e1f49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:03.819112 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da6c41-7ca2-4861-990d-585377e1f49a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:03.819328 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da6c41-7ca2-4861-990d-585377e1f49a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:03.819526 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da6c41-7ca2-4861-990d-585377e1f49a None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:03.820204 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da6c41-7ca2-4861-990d-585377e1f49a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:03.820507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da6c41-7ca2-4861-990d-585377e1f49a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:03.821884 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da6c41-7ca2-4861-990d-585377e1f49a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:03.835464 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 325/652] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:03.840687 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da6c41-7ca2-4861-990d-585377e1f49a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:03.847474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:03.949211 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da6c41-7ca2-4861-990d-585377e1f49a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:03.953448 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba9aba-711d-4a03-bae3-4f1de72f7498 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:03.991942 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a20e8-2855-423b-a49c-2a7f9e48bc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:03.992672 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a20e8-2855-423b-a49c-2a7f9e48bc0f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:03.992951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a20e8-2855-423b-a49c-2a7f9e48bc0f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:03.993234 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a20e8-2855-423b-a49c-2a7f9e48bc0f None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:03.993881 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8a20e8-2855-423b-a49c-2a7f9e48bc0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:03.994181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8a20e8-2855-423b-a49c-2a7f9e48bc0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:03.995584 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8a20e8-2855-423b-a49c-2a7f9e48bc0f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:03.997191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8720a5-a9af-4220-bcb8-96f24dede5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:03.998496 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8720a5-a9af-4220-bcb8-96f24dede5ad None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:03.998984 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8720a5-a9af-4220-bcb8-96f24dede5ad None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:03.999473 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8720a5-a9af-4220-bcb8-96f24dede5ad None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:04.000482 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8720a5-a9af-4220-bcb8-96f24dede5ad None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:04.001023 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8720a5-a9af-4220-bcb8-96f24dede5ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:04.002781 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8720a5-a9af-4220-bcb8-96f24dede5ad None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:04.009495 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 326/653] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:04.010412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a20e8-2855-423b-a49c-2a7f9e48bc0f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:04.023288 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 328/654] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Sep 14 00:52:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:04.024625 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8720a5-a9af-4220-bcb8-96f24dede5ad None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:04.052703 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:04.117908 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:04.139402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8720a5-a9af-4220-bcb8-96f24dede5ad None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:04.152403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a20e8-2855-423b-a49c-2a7f9e48bc0f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:04.177924 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c456af-1918-414e-8b6f-de89009b71fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:04.179001 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c456af-1918-414e-8b6f-de89009b71fd None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:04.179523 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c456af-1918-414e-8b6f-de89009b71fd None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:04.180033 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c456af-1918-414e-8b6f-de89009b71fd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:04.188514 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c456af-1918-414e-8b6f-de89009b71fd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:04.190420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c456af-1918-414e-8b6f-de89009b71fd None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:04.195514 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d0a20-501b-473d-a9a2-b0dac2f6f6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:04.196190 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d0a20-501b-473d-a9a2-b0dac2f6f6c5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:04.196470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d0a20-501b-473d-a9a2-b0dac2f6f6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:04.196743 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d0a20-501b-473d-a9a2-b0dac2f6f6c5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:04.205088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-b6c456af-1918-414e-8b6f-de89009b71fd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 00:52:04.205088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:04.205088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:04.205088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:04.205088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:04.205088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:04.205088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:04.205088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:04.205088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:04.205088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:04.205088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:04.205088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:04.205088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:52:04.205088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:52:04.205088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:52:04.205088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:52:04.206120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:04.206120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:04.206120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:52:04.206120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:04.206120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:52:04.206120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:04.206120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:04.206120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:04.206120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:04.206120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:04.206120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:04.206120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:04.206120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:04.206120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:04.206120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:04.206120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:04.206120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:04.206120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:04.207105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:04.207105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:04.207105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:52:04.207105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:52:04.207105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:52:04.207105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:52:04.207105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:52:04.207105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:52:04.207105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 00:52:04.207105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:04.207696 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d0a20-501b-473d-a9a2-b0dac2f6f6c5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:04.207696 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d0a20-501b-473d-a9a2-b0dac2f6f6c5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:04.210096 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 329/655] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:52:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:04.211309 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c456af-1918-414e-8b6f-de89009b71fd None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:04.223016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-c88d0a20-501b-473d-a9a2-b0dac2f6f6c5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 00:52:04.223016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:04.223016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:04.223016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:04.223016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:04.223016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:04.223016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:04.223016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:04.223016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:04.223016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:04.223016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:04.223016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:04.223016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:52:04.223016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:52:04.223016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:52:04.223016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:52:04.224399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:04.224399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:04.224399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:52:04.224399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:04.224399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:52:04.224399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:04.224399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:04.224399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:04.224399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:04.224399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:04.224399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:04.224399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:04.224399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:04.224399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:04.224399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:04.224399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:04.224399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:04.224399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:04.225375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:04.225375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:04.225375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:52:04.225375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:52:04.225375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:52:04.225375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:52:04.225375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:52:04.225375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:52:04.225375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 00:52:04.225375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:04.225375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 327/656] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 00:52:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:04.227994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d0a20-501b-473d-a9a2-b0dac2f6f6c5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:04.324458 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c456af-1918-414e-8b6f-de89009b71fd None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:04.360676 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e10e9-f983-4409-b5ef-39d04e2ab1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:04.361326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:04.361885 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e10e9-f983-4409-b5ef-39d04e2ab1e8 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:04.362469 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e10e9-f983-4409-b5ef-39d04e2ab1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:04.362945 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e10e9-f983-4409-b5ef-39d04e2ab1e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:04.377008 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:04.377259 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d0a20-501b-473d-a9a2-b0dac2f6f6c5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:04.420630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d068c-818a-4270-92a7-b7d8a0abe899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:04.421677 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d068c-818a-4270-92a7-b7d8a0abe899 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:04.422721 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d068c-818a-4270-92a7-b7d8a0abe899 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:04.423201 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d068c-818a-4270-92a7-b7d8a0abe899 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:04.424185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d068c-818a-4270-92a7-b7d8a0abe899 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:04.424699 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d068c-818a-4270-92a7-b7d8a0abe899 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:04.426358 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d068c-818a-4270-92a7-b7d8a0abe899 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:04.445457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 328/657] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Sep 14 00:52:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:04.446710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d068c-818a-4270-92a7-b7d8a0abe899 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:04.448164 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e10e9-f983-4409-b5ef-39d04e2ab1e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:04.449930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e10e9-f983-4409-b5ef-39d04e2ab1e8 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:04.567679 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d068c-818a-4270-92a7-b7d8a0abe899 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:04.573089 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 330/658] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:52:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:52:04.574018 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e10e9-f983-4409-b5ef-39d04e2ab1e8 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:04.609387 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf27cee-65a7-400f-a491-8ee82942fb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:04.610873 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf27cee-65a7-400f-a491-8ee82942fb18 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:04.611359 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf27cee-65a7-400f-a491-8ee82942fb18 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:04.611897 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf27cee-65a7-400f-a491-8ee82942fb18 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:04.613057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf27cee-65a7-400f-a491-8ee82942fb18 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:04.614435 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf27cee-65a7-400f-a491-8ee82942fb18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:04.616198 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf27cee-65a7-400f-a491-8ee82942fb18 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:04.622816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:04.639948 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 329/659] 10.222.0.42 () {56 vars in 1139 bytes} [Wed Sep 14 00:52:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:04.641822 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf27cee-65a7-400f-a491-8ee82942fb18 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:04.654582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:04.688658 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e10e9-f983-4409-b5ef-39d04e2ab1e8 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:04.725439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9233c-245e-4383-a93d-e9d6649dff28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:04.726058 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9233c-245e-4383-a93d-e9d6649dff28 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:04.726338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9233c-245e-4383-a93d-e9d6649dff28 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:04.726613 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9233c-245e-4383-a93d-e9d6649dff28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:04.755408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf27cee-65a7-400f-a491-8ee82942fb18 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:04.793606 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12649534-c6b3-44e5-b309-20551647730f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:04.794809 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12649534-c6b3-44e5-b309-20551647730f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:04.795333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12649534-c6b3-44e5-b309-20551647730f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:04.795838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12649534-c6b3-44e5-b309-20551647730f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:04.800644 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb9233c-245e-4383-a93d-e9d6649dff28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:04.802102 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb9233c-245e-4383-a93d-e9d6649dff28 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:04.808510 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12649534-c6b3-44e5-b309-20551647730f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:04.810658 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12649534-c6b3-44e5-b309-20551647730f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:04.828186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-12649534-c6b3-44e5-b309-20551647730f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:04.828186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:04.828186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:04.828186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:04.828186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:04.828186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:04.828186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:04.828186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:04.828186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:04.828186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:04.828186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:04.828186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:04.828186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:04.828186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:04.828186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:04.828186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:04.829207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:04.829207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:04.829207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:04.829207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:04.829207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:04.829207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:04.829207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:04.829207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:04.829207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:04.829207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:04.829207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:04.829207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:04.829207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:04.829207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:04.829207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:04.829207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:04.829207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:04.830133 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:04.830133 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:04.830133 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:04.832875 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 330/660] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:04.833978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12649534-c6b3-44e5-b309-20551647730f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:04.854392 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:04.913880 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:04.937440 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 331/661] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:52:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:52:04.939329 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9233c-245e-4383-a93d-e9d6649dff28 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:04.968795 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12649534-c6b3-44e5-b309-20551647730f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:05.017940 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cefb9d-6f78-4760-8954-84ba9fd590ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:05.018632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cefb9d-6f78-4760-8954-84ba9fd590ed None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:05.018880 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cefb9d-6f78-4760-8954-84ba9fd590ed None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:05.019162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cefb9d-6f78-4760-8954-84ba9fd590ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:05.041595 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9233c-245e-4383-a93d-e9d6649dff28 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:05.076705 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b6292-651b-4d82-be3f-588ea3e12ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:05.077319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b6292-651b-4d82-be3f-588ea3e12ced None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:05.077598 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b6292-651b-4d82-be3f-588ea3e12ced None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:05.077873 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b6292-651b-4d82-be3f-588ea3e12ced None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:05.079091 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b6292-651b-4d82-be3f-588ea3e12ced None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:05.079365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b6292-651b-4d82-be3f-588ea3e12ced None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:05.080905 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b6292-651b-4d82-be3f-588ea3e12ced None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:05.092678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 332/662] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:05.093493 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b6292-651b-4d82-be3f-588ea3e12ced None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:05.096854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:05.107025 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cefb9d-6f78-4760-8954-84ba9fd590ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:05.108508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cefb9d-6f78-4760-8954-84ba9fd590ed None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:05.167244 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:05.197836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b6292-651b-4d82-be3f-588ea3e12ced None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:05.212098 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 331/663] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:52:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:52:05.213364 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf46e22-6b8c-4246-b363-487938ec150d None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:05.213648 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf46e22-6b8c-4246-b363-487938ec150d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:05.213924 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf46e22-6b8c-4246-b363-487938ec150d None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:05.215311 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 332/664] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:05.220225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf46e22-6b8c-4246-b363-487938ec150d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:05.238287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61114c-664a-4c3b-90e4-349a08432095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:05.239189 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61114c-664a-4c3b-90e4-349a08432095 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:05.239735 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61114c-664a-4c3b-90e4-349a08432095 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:05.240153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61114c-664a-4c3b-90e4-349a08432095 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:05.252074 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61114c-664a-4c3b-90e4-349a08432095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:05.253662 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61114c-664a-4c3b-90e4-349a08432095 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:05.269535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-8e61114c-664a-4c3b-90e4-349a08432095 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:05.269535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:05.269535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:05.269535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:05.269535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:05.269535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:05.269535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:05.269535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:05.269535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:05.269535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:05.269535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:05.269535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:05.269535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:05.269535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:05.269535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:05.269535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:05.270528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:05.270528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:05.270528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:05.270528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:05.270528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:05.270528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:05.270528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:05.270528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:05.270528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:05.270528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:05.270528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:05.270528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:05.270528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:05.270528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:05.270528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:05.270528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:05.270528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:05.271445 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:05.271445 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:05.271445 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:05.271445 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 333/665] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:05.271445 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61114c-664a-4c3b-90e4-349a08432095 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:05.329783 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:05.330045 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf46e22-6b8c-4246-b363-487938ec150d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:05.365683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93bcf0-70ef-4dc7-bdce-6fec3d884d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:05.366301 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93bcf0-70ef-4dc7-bdce-6fec3d884d89 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:05.366581 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93bcf0-70ef-4dc7-bdce-6fec3d884d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:05.366855 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93bcf0-70ef-4dc7-bdce-6fec3d884d89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:05.373590 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93bcf0-70ef-4dc7-bdce-6fec3d884d89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:05.375038 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93bcf0-70ef-4dc7-bdce-6fec3d884d89 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:05.379776 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61114c-664a-4c3b-90e4-349a08432095 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:05.387543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-3e93bcf0-70ef-4dc7-bdce-6fec3d884d89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:05.387543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:05.387543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:05.387543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:05.387543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:05.387543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:05.387543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:05.387543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:05.387543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:05.387543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:05.387543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:05.387543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:05.387543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:52:05.387543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:52:05.387543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:52:05.388449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:52:05.388449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:05.388449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:05.388449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:05.388449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:05.388449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:05.388449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:05.388449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:05.388449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:05.388449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:05.388449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:05.388449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:05.388449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:05.388449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:05.388449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:05.388449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:05.388449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:05.389409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:52:05.389409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:52:05.389409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:52:05.389409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:52:05.389409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:52:05.389409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:52:05.389409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:05.389409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:05.389409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 333/666] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:05.390086 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8a208-e213-4c3f-a3d7-951db023ddd0 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:05.390370 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8a208-e213-4c3f-a3d7-951db023ddd0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:05.390958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8a208-e213-4c3f-a3d7-951db023ddd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:05.410036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2ea8a208-e213-4c3f-a3d7-951db023ddd0 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:05.411265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2ea8a208-e213-4c3f-a3d7-951db023ddd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:05.421494 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ab905-776c-4a52-875f-a074028e339e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:05.423113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ab905-776c-4a52-875f-a074028e339e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:05.423711 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ab905-776c-4a52-875f-a074028e339e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:05.424313 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ab905-776c-4a52-875f-a074028e339e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:05.428212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:05.439162 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3ab905-776c-4a52-875f-a074028e339e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:05.441131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3ab905-776c-4a52-875f-a074028e339e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:05.457539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-cd3ab905-776c-4a52-875f-a074028e339e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:05.457539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:05.457539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:05.457539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:05.457539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:05.457539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:05.457539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:05.457539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:05.457539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:05.457539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:05.457539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:05.457539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:05.457539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:05.457539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:05.457539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:05.457539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:05.458600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:05.458600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:05.458600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:05.458600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:05.458600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:05.458600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:05.458600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:05.458600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:05.458600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:05.458600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:05.458600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:05.458600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:05.458600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:05.458600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:05.458600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:05.458600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:05.458600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:05.459738 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:05.459738 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:05.459738 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:05.460975 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 334/667] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:05.461802 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ab905-776c-4a52-875f-a074028e339e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:05.491912 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 334/668] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:52:05.501170 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8a208-e213-4c3f-a3d7-951db023ddd0 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:05.570930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ab905-776c-4a52-875f-a074028e339e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:05.578495 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:05.607896 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3df53-36b6-443b-84c2-c616a37cf8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:05.608504 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3df53-36b6-443b-84c2-c616a37cf8ff None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:05.608826 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3df53-36b6-443b-84c2-c616a37cf8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:05.609106 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3df53-36b6-443b-84c2-c616a37cf8ff None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:05.609720 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe3df53-36b6-443b-84c2-c616a37cf8ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:05.610026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe3df53-36b6-443b-84c2-c616a37cf8ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:05.611460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe3df53-36b6-443b-84c2-c616a37cf8ff None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:05.621451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8a208-e213-4c3f-a3d7-951db023ddd0 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:05.622912 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 335/669] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:05.627625 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3df53-36b6-443b-84c2-c616a37cf8ff None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:05.664364 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990c1ae-a39f-442a-912a-9811ab168b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:05.664976 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990c1ae-a39f-442a-912a-9811ab168b1d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:05.665255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990c1ae-a39f-442a-912a-9811ab168b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:05.665530 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990c1ae-a39f-442a-912a-9811ab168b1d None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:05.666218 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9990c1ae-a39f-442a-912a-9811ab168b1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:05.666526 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9990c1ae-a39f-442a-912a-9811ab168b1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:05.668416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9990c1ae-a39f-442a-912a-9811ab168b1d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:05.685014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 335/670] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:52:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:05.685014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a64124-a2c9-412c-b931-ed86853b0c11 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:05.685014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a64124-a2c9-412c-b931-ed86853b0c11 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:05.685014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a64124-a2c9-412c-b931-ed86853b0c11 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:05.685014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 336/671] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:05.685014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a64124-a2c9-412c-b931-ed86853b0c11 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:05.696742 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:05.740550 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3df53-36b6-443b-84c2-c616a37cf8ff None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:05.775689 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ac941-daee-4be7-9790-82fe7fa53d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:05.776301 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ac941-daee-4be7-9790-82fe7fa53d50 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:05.776581 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ac941-daee-4be7-9790-82fe7fa53d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:05.776856 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ac941-daee-4be7-9790-82fe7fa53d50 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:05.777497 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ac941-daee-4be7-9790-82fe7fa53d50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:05.777800 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ac941-daee-4be7-9790-82fe7fa53d50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:05.779181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ac941-daee-4be7-9790-82fe7fa53d50 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:05.796638 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 336/672] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:05.796638 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ac941-daee-4be7-9790-82fe7fa53d50 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:05.798690 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a64124-a2c9-412c-b931-ed86853b0c11 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:05.821843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:05.831937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f082079-dc27-4bee-b482-d0002b4427a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:05.832564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f082079-dc27-4bee-b482-d0002b4427a0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:05.832844 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f082079-dc27-4bee-b482-d0002b4427a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:05.833115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f082079-dc27-4bee-b482-d0002b4427a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:05.839977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f082079-dc27-4bee-b482-d0002b4427a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:05.841422 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f082079-dc27-4bee-b482-d0002b4427a0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:05.852925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-0f082079-dc27-4bee-b482-d0002b4427a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:05.852925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:05.852925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:05.852925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:05.852925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:05.852925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:05.852925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:05.852925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:05.852925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:05.852925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:05.852925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:05.852925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:05.852925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:52:05.852925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:52:05.852925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:52:05.853994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:52:05.853994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:05.853994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:05.853994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:05.853994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:05.853994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:05.853994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:05.853994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:05.853994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:05.853994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:05.853994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:05.853994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:05.853994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:05.853994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:05.853994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:05.853994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:05.853994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:05.854888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:52:05.854888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:52:05.854888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:52:05.854888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:52:05.854888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:52:05.854888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:52:05.854888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:05.854888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:05.854888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 337/673] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:05.855775 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67b7e1-8e3e-4ac6-8dc7-d14a661839e2 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:05.855775 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67b7e1-8e3e-4ac6-8dc7-d14a661839e2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:05.855775 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67b7e1-8e3e-4ac6-8dc7-d14a661839e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:05.873278 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-3b67b7e1-8e3e-4ac6-8dc7-d14a661839e2 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:05.874455 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3b67b7e1-8e3e-4ac6-8dc7-d14a661839e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:05.900140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ac941-daee-4be7-9790-82fe7fa53d50 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:05.933104 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe26eea-c21b-4dd6-a61b-b28e955783d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:05.933707 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe26eea-c21b-4dd6-a61b-b28e955783d0 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:05.933984 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe26eea-c21b-4dd6-a61b-b28e955783d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:05.934254 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe26eea-c21b-4dd6-a61b-b28e955783d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:05.934974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe26eea-c21b-4dd6-a61b-b28e955783d0 None admin] RBAC: Adding query filter params (role.id=7a33c2ceb92a478f81753573f8ece1bd, scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3, user.id=89ff357209194276bc079d0c05898747) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:05.935329 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe26eea-c21b-4dd6-a61b-b28e955783d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:05.935869 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:05.936810 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe26eea-c21b-4dd6-a61b-b28e955783d0 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:05.940969 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 338/674] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:52:05.941890 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67b7e1-8e3e-4ac6-8dc7-d14a661839e2 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:05.949766 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 337/675] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:52:05 2022] GET /identity/v3/role_assignments?user.id=89ff357209194276bc079d0c05898747&scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3&role.id=7a33c2ceb92a478f81753573f8ece1bd => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:05.949766 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe26eea-c21b-4dd6-a61b-b28e955783d0 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:06.066528 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67b7e1-8e3e-4ac6-8dc7-d14a661839e2 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:06.089970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:06.105663 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe26eea-c21b-4dd6-a61b-b28e955783d0 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:06.113461 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915b764-1106-4862-9917-0bfced56da08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:06.114117 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915b764-1106-4862-9917-0bfced56da08 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:06.114397 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915b764-1106-4862-9917-0bfced56da08 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:06.114669 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915b764-1106-4862-9917-0bfced56da08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:06.129649 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915b764-1106-4862-9917-0bfced56da08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:52:06.136527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-b915b764-1106-4862-9917-0bfced56da08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:52:06.136527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:06.136527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:06.136527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:06.136527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:06.136527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:06.136527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:06.136527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:06.136527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:06.136527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:06.136527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:06.136527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:06.136527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:52:06.136527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:52:06.136527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:52:06.136527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:52:06.137520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:52:06.137520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:52:06.137520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:52:06.137520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:52:06.137520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:06.137520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:06.137520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:06.137520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:06.137520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:06.137520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:06.137520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:06.137520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:06.137520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:06.137520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:06.137520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:06.137520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:06.138377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:06.138377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:06.138377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:06.138377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:06.138377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:52:06.138377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:52:06.138377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:52:06.138377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:52:06.138377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:52:06.138377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:52:06.138377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:52:06.138377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:06.138377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 339/676] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:52:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:06.139904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915b764-1106-4862-9917-0bfced56da08 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:06.156240 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d131a7-6b28-47a8-99fb-67c2c7dc9f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:06.156741 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d131a7-6b28-47a8-99fb-67c2c7dc9f7d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:06.157022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d131a7-6b28-47a8-99fb-67c2c7dc9f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:06.157371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d131a7-6b28-47a8-99fb-67c2c7dc9f7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:06.167137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d131a7-6b28-47a8-99fb-67c2c7dc9f7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:06.168633 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d131a7-6b28-47a8-99fb-67c2c7dc9f7d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:06.181502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-70d131a7-6b28-47a8-99fb-67c2c7dc9f7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:06.181502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:06.181502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:06.181502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:06.181502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:06.181502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:06.181502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:06.181502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:06.181502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:06.181502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:06.181502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:06.181502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:06.181502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:52:06.181502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:52:06.181502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:52:06.182431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:52:06.182431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:06.182431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:06.182431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:06.182431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:06.182431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:06.182431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:06.182431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:06.182431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:06.182431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:06.182431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:06.182431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:06.182431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:06.182431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:06.182431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:06.182431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:06.182431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:06.183352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:52:06.183352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:52:06.183352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:52:06.183352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:52:06.183352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:52:06.183352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:52:06.183352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:06.183352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:06.187435 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 338/677] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:06.188995 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d131a7-6b28-47a8-99fb-67c2c7dc9f7d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:06.215921 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:06.270006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915b764-1106-4862-9917-0bfced56da08 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:06.301947 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d131a7-6b28-47a8-99fb-67c2c7dc9f7d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:06.311820 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816b14d-38e5-408c-bae8-f1ef3f11cfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:06.313644 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816b14d-38e5-408c-bae8-f1ef3f11cfb9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:06.314333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816b14d-38e5-408c-bae8-f1ef3f11cfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:06.314904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816b14d-38e5-408c-bae8-f1ef3f11cfb9 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:06.315829 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816b14d-38e5-408c-bae8-f1ef3f11cfb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:06.316370 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816b14d-38e5-408c-bae8-f1ef3f11cfb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:06.318106 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816b14d-38e5-408c-bae8-f1ef3f11cfb9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:06.337970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367131d-ccd3-4168-9555-48e6a804e0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:06.338192 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 340/678] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:52:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:06.338524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367131d-ccd3-4168-9555-48e6a804e0d5 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:06.338805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367131d-ccd3-4168-9555-48e6a804e0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:06.338992 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816b14d-38e5-408c-bae8-f1ef3f11cfb9 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:06.339089 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367131d-ccd3-4168-9555-48e6a804e0d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:06.339810 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6367131d-ccd3-4168-9555-48e6a804e0d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:06.340057 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6367131d-ccd3-4168-9555-48e6a804e0d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:06.341420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6367131d-ccd3-4168-9555-48e6a804e0d5 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:06.351519 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 339/679] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:52:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:06.352255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367131d-ccd3-4168-9555-48e6a804e0d5 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:06.369107 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:06.463362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367131d-ccd3-4168-9555-48e6a804e0d5 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:06.465710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816b14d-38e5-408c-bae8-f1ef3f11cfb9 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:06.492503 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:06.503914 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4afa2-073a-4272-9971-950e7389610a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:06.504550 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4afa2-073a-4272-9971-950e7389610a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:06.504828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4afa2-073a-4272-9971-950e7389610a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:06.505147 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4afa2-073a-4272-9971-950e7389610a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:06.509483 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aefc676-c8da-4c2e-971c-6b9dedcb9560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:06.510389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefc676-c8da-4c2e-971c-6b9dedcb9560 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:06.510672 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefc676-c8da-4c2e-971c-6b9dedcb9560 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:06.510935 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aefc676-c8da-4c2e-971c-6b9dedcb9560 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:06.511658 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aefc676-c8da-4c2e-971c-6b9dedcb9560 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:06.511966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aefc676-c8da-4c2e-971c-6b9dedcb9560 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:06.513400 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aefc676-c8da-4c2e-971c-6b9dedcb9560 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:06.516071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a4afa2-073a-4272-9971-950e7389610a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:52:06.522260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-90a4afa2-073a-4272-9971-950e7389610a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:52:06.522260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:06.522260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:06.522260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:06.522260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:06.522260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:06.522260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:06.522260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:06.522260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:06.522260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:06.522260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:06.522260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:06.522260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:52:06.522260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:52:06.522260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:52:06.522260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:52:06.523182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:52:06.523182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:52:06.523182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:52:06.523182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:52:06.523182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:06.523182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:06.523182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:06.523182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:06.523182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:06.523182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:06.523182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:06.523182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:06.523182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:06.523182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:06.523182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:06.523182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:06.524396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:06.524396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:06.524396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:06.524396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:06.524396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:52:06.524396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:52:06.524396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:52:06.524396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:52:06.524396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:52:06.524396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:52:06.524396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:52:06.524396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:06.525851 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 341/680] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:52:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:06.526680 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aefc676-c8da-4c2e-971c-6b9dedcb9560 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:06.534323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 340/681] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 00:52:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:06.534323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4afa2-073a-4272-9971-950e7389610a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:06.610735 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:06.639583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aefc676-c8da-4c2e-971c-6b9dedcb9560 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:06.655578 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4afa2-073a-4272-9971-950e7389610a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:06.692757 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347af67-7997-4f1e-8fa9-08b2179415e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:06.699578 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347af67-7997-4f1e-8fa9-08b2179415e8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:06.699578 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347af67-7997-4f1e-8fa9-08b2179415e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:06.699578 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347af67-7997-4f1e-8fa9-08b2179415e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:06.699578 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347af67-7997-4f1e-8fa9-08b2179415e8 None admin] RBAC: Adding query filter params (role.id=7a33c2ceb92a478f81753573f8ece1bd, scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3, user.id=9373aa297439487691fae325c0245bb9) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:06.699955 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd625eaf-5fae-4bd2-9670-a14aed040b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:06.700068 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347af67-7997-4f1e-8fa9-08b2179415e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:06.700197 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd625eaf-5fae-4bd2-9670-a14aed040b20 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:06.700197 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd625eaf-5fae-4bd2-9670-a14aed040b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:06.700197 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd625eaf-5fae-4bd2-9670-a14aed040b20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:06.700440 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347af67-7997-4f1e-8fa9-08b2179415e8 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:06.705333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 342/682] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:52:06 2022] GET /identity/v3/role_assignments?user.id=9373aa297439487691fae325c0245bb9&scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3&role.id=7a33c2ceb92a478f81753573f8ece1bd => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:06.705511 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347af67-7997-4f1e-8fa9-08b2179415e8 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:06.731710 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd625eaf-5fae-4bd2-9670-a14aed040b20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:06.733646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd625eaf-5fae-4bd2-9670-a14aed040b20 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:06.772349 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-fd625eaf-5fae-4bd2-9670-a14aed040b20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:06.772349 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:06.772349 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:06.772349 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:06.772349 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:06.772349 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:06.772349 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:06.772349 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:06.772349 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:06.772349 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:06.772349 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:06.772349 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:06.772349 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:06.772349 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:06.772349 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:06.772349 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:06.773386 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:06.773386 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:06.773386 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:06.773386 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:06.773386 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:06.773386 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:06.773386 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:06.773386 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:06.773386 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:06.773386 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:06.773386 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:06.773386 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:06.773386 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:06.773386 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:06.773386 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:06.773386 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:06.773386 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:06.774313 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:06.774313 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:06.774313 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:06.783924 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 341/683] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:06.784996 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd625eaf-5fae-4bd2-9670-a14aed040b20 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:06.798782 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:06.864711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347af67-7997-4f1e-8fa9-08b2179415e8 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:06.914036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68870ce-e8a0-4386-a207-89a1f4e4e733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:06.915017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68870ce-e8a0-4386-a207-89a1f4e4e733 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:06.915298 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68870ce-e8a0-4386-a207-89a1f4e4e733 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:06.915979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68870ce-e8a0-4386-a207-89a1f4e4e733 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:06.918857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68870ce-e8a0-4386-a207-89a1f4e4e733 None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:06.918857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68870ce-e8a0-4386-a207-89a1f4e4e733 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:06.919038 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68870ce-e8a0-4386-a207-89a1f4e4e733 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:06.930566 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 343/684] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 00:52:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:06.932922 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68870ce-e8a0-4386-a207-89a1f4e4e733 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:06.999777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd625eaf-5fae-4bd2-9670-a14aed040b20 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:07.058234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:07.085134 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ba1ff-3f17-44b6-b0bd-e210db757dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:07.086288 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ba1ff-3f17-44b6-b0bd-e210db757dec None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:07.087255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ba1ff-3f17-44b6-b0bd-e210db757dec None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:07.087924 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ba1ff-3f17-44b6-b0bd-e210db757dec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:07.088945 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ba1ff-3f17-44b6-b0bd-e210db757dec None admin] RBAC: Adding query filter params (role.id=7a33c2ceb92a478f81753573f8ece1bd, scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3, user.id=4d64f9bca7c546e4ba7f54c1c47d665e) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:07.090404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ba1ff-3f17-44b6-b0bd-e210db757dec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:07.099057 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ba1ff-3f17-44b6-b0bd-e210db757dec None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:07.116727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 342/685] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:52:06 2022] GET /identity/v3/role_assignments?user.id=4d64f9bca7c546e4ba7f54c1c47d665e&scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3&role.id=7a33c2ceb92a478f81753573f8ece1bd => generated 608 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:07.117603 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ba1ff-3f17-44b6-b0bd-e210db757dec None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:07.128999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68870ce-e8a0-4386-a207-89a1f4e4e733 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:07.146249 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:07.211315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953fca6-c12f-4048-8c0f-aa8e4b0496b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:07.212404 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953fca6-c12f-4048-8c0f-aa8e4b0496b2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:07.212912 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953fca6-c12f-4048-8c0f-aa8e4b0496b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:07.213389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953fca6-c12f-4048-8c0f-aa8e4b0496b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:07.214248 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953fca6-c12f-4048-8c0f-aa8e4b0496b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:07.214767 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953fca6-c12f-4048-8c0f-aa8e4b0496b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:07.216811 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953fca6-c12f-4048-8c0f-aa8e4b0496b2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:07.242788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 344/686] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:07.248517 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953fca6-c12f-4048-8c0f-aa8e4b0496b2 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:07.348332 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ba1ff-3f17-44b6-b0bd-e210db757dec None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:07.381272 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953fca6-c12f-4048-8c0f-aa8e4b0496b2 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:07.385630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7d7da-08a4-4d58-918e-d9c1ae50be7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:07.386275 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7d7da-08a4-4d58-918e-d9c1ae50be7c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:07.386560 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7d7da-08a4-4d58-918e-d9c1ae50be7c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:07.386837 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7d7da-08a4-4d58-918e-d9c1ae50be7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:07.402190 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e7d7da-08a4-4d58-918e-d9c1ae50be7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:07.403788 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e7d7da-08a4-4d58-918e-d9c1ae50be7c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:07.419253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-d5e7d7da-08a4-4d58-918e-d9c1ae50be7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:07.419253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:07.419253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:07.419253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:07.419253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:07.419253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:07.419253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:07.419253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:07.419253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:07.419253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:07.419253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:07.419253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:07.419253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:07.419253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:07.419253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:07.419253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:07.420438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:07.420438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:07.420438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:07.420438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:07.420438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:07.420438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:07.420438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:07.420438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:07.420438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:07.420438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:07.420438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:07.420438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:07.420438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:07.420438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:07.420438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:07.420438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:07.420438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:07.421376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:07.421376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:07.421376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:07.421549 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 343/687] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:07.424764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6172521a-1379-4c17-9323-8797121b97f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:07.425675 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172521a-1379-4c17-9323-8797121b97f4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:07.426170 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172521a-1379-4c17-9323-8797121b97f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:07.426646 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172521a-1379-4c17-9323-8797121b97f4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:07.427508 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7d7da-08a4-4d58-918e-d9c1ae50be7c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:07.435377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6172521a-1379-4c17-9323-8797121b97f4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:07.437164 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6172521a-1379-4c17-9323-8797121b97f4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:07.450500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-6172521a-1379-4c17-9323-8797121b97f4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 00:52:07.450500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:07.450500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:07.450500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:07.450500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:07.450500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:07.450500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:07.450500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:07.450500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:07.450500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:07.450500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:07.450500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:07.450500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:52:07.450500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:52:07.450500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:52:07.450500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:52:07.451491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:07.451491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:07.451491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:52:07.451491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:07.451491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:52:07.451491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:07.451491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:07.451491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:07.451491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:07.451491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:07.451491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:07.451491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:07.451491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:07.451491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:07.451491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:07.451491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:07.451491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:07.451491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:07.452760 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:07.452760 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:07.452760 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:52:07.452760 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:52:07.452760 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:52:07.452760 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:52:07.452760 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:52:07.452760 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:52:07.452760 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 00:52:07.452760 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:07.455722 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 345/688] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Sep 14 00:52:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:52:07.462434 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6172521a-1379-4c17-9323-8797121b97f4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:07.487765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:07.497381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:07.556975 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7d7da-08a4-4d58-918e-d9c1ae50be7c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:07.598855 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6172521a-1379-4c17-9323-8797121b97f4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:07.599274 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b346427-0bb4-4fcc-974e-5c34e2e9008d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:07.599939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b346427-0bb4-4fcc-974e-5c34e2e9008d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:07.600223 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b346427-0bb4-4fcc-974e-5c34e2e9008d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:07.600494 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b346427-0bb4-4fcc-974e-5c34e2e9008d None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:07.601150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b346427-0bb4-4fcc-974e-5c34e2e9008d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:07.601458 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b346427-0bb4-4fcc-974e-5c34e2e9008d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:07.604626 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b346427-0bb4-4fcc-974e-5c34e2e9008d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:07.618988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 344/689] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:07.624367 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b346427-0bb4-4fcc-974e-5c34e2e9008d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:07.649714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b190b39-249a-40e6-9dde-96e5166905f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:07.650478 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b190b39-249a-40e6-9dde-96e5166905f1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:07.650855 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b190b39-249a-40e6-9dde-96e5166905f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:07.651229 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b190b39-249a-40e6-9dde-96e5166905f1 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:07.652191 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b190b39-249a-40e6-9dde-96e5166905f1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:07.652596 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b190b39-249a-40e6-9dde-96e5166905f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:07.654336 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b190b39-249a-40e6-9dde-96e5166905f1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:07.679584 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 346/690] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Sep 14 00:52:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:07.682489 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b190b39-249a-40e6-9dde-96e5166905f1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:07.776974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:07.796291 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b190b39-249a-40e6-9dde-96e5166905f1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:07.814002 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:07.821506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b346427-0bb4-4fcc-974e-5c34e2e9008d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:07.836713 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe1707-f953-4657-9bdd-958bed8a7bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:07.837676 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe1707-f953-4657-9bdd-958bed8a7bba None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:07.838165 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe1707-f953-4657-9bdd-958bed8a7bba None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:07.838665 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe1707-f953-4657-9bdd-958bed8a7bba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:07.916623 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe1707-f953-4657-9bdd-958bed8a7bba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:07.918448 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe1707-f953-4657-9bdd-958bed8a7bba None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:07.925174 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a2596-8c16-458a-94af-60917409a02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:07.925828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a2596-8c16-458a-94af-60917409a02c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:07.926107 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a2596-8c16-458a-94af-60917409a02c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:07.926370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a2596-8c16-458a-94af-60917409a02c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:07.942054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a2596-8c16-458a-94af-60917409a02c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:07.943593 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a2596-8c16-458a-94af-60917409a02c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:07.972266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-2a8a2596-8c16-458a-94af-60917409a02c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 00:52:07.972266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:07.972266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:07.972266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:07.972266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:07.972266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:07.972266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:07.972266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:07.972266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:07.972266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:07.972266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:07.972266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:07.972266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:52:07.972266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:52:07.972266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:52:07.972266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:52:07.973290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:07.973290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:07.973290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:52:07.973290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:07.973290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:52:07.973290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:07.973290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:07.973290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:07.973290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:07.973290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:07.973290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:07.973290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:07.973290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:07.973290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:07.973290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:07.973290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:07.973290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:07.973290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:07.974260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:07.974260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:07.974260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:52:07.974260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:52:07.974260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:52:07.974260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:52:07.974260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:52:07.974260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:52:07.974260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 00:52:07.974260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:07.974260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 345/691] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:52:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:52:07.988201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a2596-8c16-458a-94af-60917409a02c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:08.022318 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:08.041660 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 347/692] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:52:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:52:08.047957 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe1707-f953-4657-9bdd-958bed8a7bba None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:08.155573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe1707-f953-4657-9bdd-958bed8a7bba None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:08.192427 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93530100-32b7-454d-8eed-fc6b7cad6932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:08.193491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93530100-32b7-454d-8eed-fc6b7cad6932 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:08.194047 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93530100-32b7-454d-8eed-fc6b7cad6932 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:08.194545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93530100-32b7-454d-8eed-fc6b7cad6932 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:08.206030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93530100-32b7-454d-8eed-fc6b7cad6932 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:08.207796 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93530100-32b7-454d-8eed-fc6b7cad6932 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:08.209446 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a2596-8c16-458a-94af-60917409a02c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:08.224531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-93530100-32b7-454d-8eed-fc6b7cad6932 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:08.224531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:08.224531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:08.224531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:08.224531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:08.224531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:08.224531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:08.224531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:08.224531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:08.224531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:08.224531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:08.224531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:08.224531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:08.224531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:08.224531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:08.224531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:08.225862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:08.225862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:08.225862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:08.225862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:08.225862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:08.225862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:08.225862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:08.225862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:08.225862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:08.225862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:08.225862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:08.225862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:08.225862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:08.225862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:08.225862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:08.225862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:08.225862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:08.226786 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:08.226786 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:08.226786 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:08.228984 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 348/693] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:52:08.234842 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93530100-32b7-454d-8eed-fc6b7cad6932 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:08.273868 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:08.300425 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720870b7-ebcd-4f94-a498-a4adce43252e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:08.301098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720870b7-ebcd-4f94-a498-a4adce43252e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:08.301354 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720870b7-ebcd-4f94-a498-a4adce43252e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:08.301624 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720870b7-ebcd-4f94-a498-a4adce43252e None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:08.302388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720870b7-ebcd-4f94-a498-a4adce43252e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:08.302697 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720870b7-ebcd-4f94-a498-a4adce43252e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:08.304246 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720870b7-ebcd-4f94-a498-a4adce43252e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:08.336939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 346/694] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Sep 14 00:52:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:08.342628 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720870b7-ebcd-4f94-a498-a4adce43252e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:08.375927 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93530100-32b7-454d-8eed-fc6b7cad6932 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:08.386626 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:08.431327 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eeace3-09d4-425b-a93d-e92b87bb1bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:08.432326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eeace3-09d4-425b-a93d-e92b87bb1bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:08.432921 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eeace3-09d4-425b-a93d-e92b87bb1bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:08.433419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eeace3-09d4-425b-a93d-e92b87bb1bd5 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:08.434251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eeace3-09d4-425b-a93d-e92b87bb1bd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:08.434752 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eeace3-09d4-425b-a93d-e92b87bb1bd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:08.436242 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eeace3-09d4-425b-a93d-e92b87bb1bd5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:08.448993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 349/695] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:08.450233 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a968b8-a4f8-4740-b7d8-079f1ecc255d None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:08.455599 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a968b8-a4f8-4740-b7d8-079f1ecc255d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:08.455990 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a968b8-a4f8-4740-b7d8-079f1ecc255d None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:08.461421 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 350/696] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:08.461421 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a968b8-a4f8-4740-b7d8-079f1ecc255d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:08.505197 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720870b7-ebcd-4f94-a498-a4adce43252e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:08.546546 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364d510-dd3f-4709-bc05-638184c82da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:08.548014 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364d510-dd3f-4709-bc05-638184c82da2 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:08.548480 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364d510-dd3f-4709-bc05-638184c82da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:08.549110 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364d510-dd3f-4709-bc05-638184c82da2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:08.579118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:08.587148 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a968b8-a4f8-4740-b7d8-079f1ecc255d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:08.624011 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc45f2a-a1be-49f4-832f-e1f3e0203078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:08.624789 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc45f2a-a1be-49f4-832f-e1f3e0203078 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:08.625810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc45f2a-a1be-49f4-832f-e1f3e0203078 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:08.626281 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc45f2a-a1be-49f4-832f-e1f3e0203078 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:08.633454 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc45f2a-a1be-49f4-832f-e1f3e0203078 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:08.635078 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc45f2a-a1be-49f4-832f-e1f3e0203078 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:08.646179 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364d510-dd3f-4709-bc05-638184c82da2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:08.647266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-bdc45f2a-a1be-49f4-832f-e1f3e0203078 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:08.647266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:08.647266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:08.647266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:08.647266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:08.647266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:08.647266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:08.647266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:08.647266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:08.647266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:08.647266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:08.647266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:08.647266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:52:08.647266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:52:08.647266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:52:08.648315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:52:08.648315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:08.648315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:08.648315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:08.648315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:08.648315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:08.648315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:08.648315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:08.648315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:08.648315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:08.648315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:08.648315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:08.648315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:08.648315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:08.648315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:08.648315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:08.648315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:08.649438 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:52:08.649438 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:52:08.649438 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:52:08.649438 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:52:08.649438 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:52:08.649438 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:52:08.649438 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:08.649438 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:08.650601 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364d510-dd3f-4709-bc05-638184c82da2 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:08.655695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 351/697] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:08.657650 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb404923-767a-42b8-a373-46f3302c8f0c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:08.658035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb404923-767a-42b8-a373-46f3302c8f0c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:08.658585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb404923-767a-42b8-a373-46f3302c8f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:08.673989 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:08.679425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-cb404923-767a-42b8-a373-46f3302c8f0c None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:08.680770 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cb404923-767a-42b8-a373-46f3302c8f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:08.760277 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 352/698] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:52:08.767644 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb404923-767a-42b8-a373-46f3302c8f0c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:08.797924 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 347/699] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:52:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 00:52:08.804388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364d510-dd3f-4709-bc05-638184c82da2 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:08.910993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:08.933351 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb404923-767a-42b8-a373-46f3302c8f0c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:08.969463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364d510-dd3f-4709-bc05-638184c82da2 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:08.995817 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:08.998474 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133526bb-a97e-4d45-b8ea-cb005b12385d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:08.999224 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133526bb-a97e-4d45-b8ea-cb005b12385d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:08.999440 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133526bb-a97e-4d45-b8ea-cb005b12385d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:08.999747 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133526bb-a97e-4d45-b8ea-cb005b12385d None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:09.000644 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133526bb-a97e-4d45-b8ea-cb005b12385d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:09.000951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133526bb-a97e-4d45-b8ea-cb005b12385d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:09.002391 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133526bb-a97e-4d45-b8ea-cb005b12385d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:09.018050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 353/700] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:52:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:09.025582 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682daa7-4ffc-40c9-ae50-0701923fe03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:09.026221 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682daa7-4ffc-40c9-ae50-0701923fe03b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:09.026500 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682daa7-4ffc-40c9-ae50-0701923fe03b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:09.026773 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682daa7-4ffc-40c9-ae50-0701923fe03b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:09.028743 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133526bb-a97e-4d45-b8ea-cb005b12385d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:09.041517 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682daa7-4ffc-40c9-ae50-0701923fe03b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:09.042998 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682daa7-4ffc-40c9-ae50-0701923fe03b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:09.070409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-8682daa7-4ffc-40c9-ae50-0701923fe03b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:09.070409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:09.070409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:09.070409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:09.070409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:09.070409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:09.070409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:09.070409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:09.070409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:09.070409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:09.070409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:09.070409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:09.070409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:09.070409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:09.070409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:09.070409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:09.071394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:09.071394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:09.071394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:09.071394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:09.071394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:09.071394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:09.071394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:09.071394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:09.071394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:09.071394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:09.071394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:09.071394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:09.071394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:09.071394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:09.071394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:09.071394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:09.071394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:09.072458 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:09.072458 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:09.072458 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:09.074060 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 348/701] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:09.079246 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682daa7-4ffc-40c9-ae50-0701923fe03b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:09.174437 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133526bb-a97e-4d45-b8ea-cb005b12385d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:09.204632 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682daa7-4ffc-40c9-ae50-0701923fe03b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:09.214844 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73349e94-7dac-4f9f-9bdc-1268ffbf2d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:09.215478 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73349e94-7dac-4f9f-9bdc-1268ffbf2d89 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:09.215881 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73349e94-7dac-4f9f-9bdc-1268ffbf2d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:09.216151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73349e94-7dac-4f9f-9bdc-1268ffbf2d89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:09.216993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73349e94-7dac-4f9f-9bdc-1268ffbf2d89 None admin] RBAC: Adding query filter params (role.id=ada268ea63354a0297070369dfe3c6cc, scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3, user.id=5e0fe2e84f6748c992c9beedf2f5d49b) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:09.217209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73349e94-7dac-4f9f-9bdc-1268ffbf2d89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:09.218688 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73349e94-7dac-4f9f-9bdc-1268ffbf2d89 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:09.226253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 354/702] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:52:09 2022] GET /identity/v3/role_assignments?user.id=5e0fe2e84f6748c992c9beedf2f5d49b&scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3&role.id=ada268ea63354a0297070369dfe3c6cc => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:09.227132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73349e94-7dac-4f9f-9bdc-1268ffbf2d89 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:09.243804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:09.263589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1ed00-814b-47b7-b235-be826662e252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:09.265454 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1ed00-814b-47b7-b235-be826662e252 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:09.266195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1ed00-814b-47b7-b235-be826662e252 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:09.266608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1ed00-814b-47b7-b235-be826662e252 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:09.297125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c1ed00-814b-47b7-b235-be826662e252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:09.301131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c1ed00-814b-47b7-b235-be826662e252 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:09.340011 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-53c1ed00-814b-47b7-b235-be826662e252 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:09.340011 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:09.340011 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:09.340011 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:09.340011 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:09.340011 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:09.340011 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:09.340011 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:09.340011 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:09.340011 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:09.340011 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:09.340011 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:09.340011 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:09.340011 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:09.340011 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:09.340011 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:09.341028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:09.341028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:09.341028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:09.341028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:09.341028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:09.341028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:09.341028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:09.341028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:09.341028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:09.341028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:09.341028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:09.341028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:09.341028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:09.341028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:09.341028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:09.341028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:09.341028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:09.342544 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:09.342544 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:09.342544 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:09.342544 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 349/703] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:09.355272 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1ed00-814b-47b7-b235-be826662e252 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:09.387611 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:09.419002 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73349e94-7dac-4f9f-9bdc-1268ffbf2d89 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:09.461317 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a148be-da86-4e20-9b23-094ed8b32d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:09.461936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a148be-da86-4e20-9b23-094ed8b32d69 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:09.462222 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a148be-da86-4e20-9b23-094ed8b32d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:09.462497 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a148be-da86-4e20-9b23-094ed8b32d69 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:09.463141 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a148be-da86-4e20-9b23-094ed8b32d69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:09.463445 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a148be-da86-4e20-9b23-094ed8b32d69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:09.464901 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a148be-da86-4e20-9b23-094ed8b32d69 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:09.488223 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 355/704] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:09.488223 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51f81c-dd16-4ca0-92ed-bfb4734a7eab None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:09.488223 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51f81c-dd16-4ca0-92ed-bfb4734a7eab None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:09.488223 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51f81c-dd16-4ca0-92ed-bfb4734a7eab None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:09.488595 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1ed00-814b-47b7-b235-be826662e252 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:09.491800 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 356/705] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:09.492773 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51f81c-dd16-4ca0-92ed-bfb4734a7eab None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:09.530454 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872629e-1e65-48c3-83b9-b8965f191ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:09.531865 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872629e-1e65-48c3-83b9-b8965f191ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:09.532407 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872629e-1e65-48c3-83b9-b8965f191ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:09.533067 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872629e-1e65-48c3-83b9-b8965f191ff9 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:09.533817 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872629e-1e65-48c3-83b9-b8965f191ff9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:09.534222 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872629e-1e65-48c3-83b9-b8965f191ff9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:09.535740 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d872629e-1e65-48c3-83b9-b8965f191ff9 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:09.555663 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 350/706] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:09.555663 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33387649-8b70-4e25-b664-4cb158212143 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:09.555663 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33387649-8b70-4e25-b664-4cb158212143 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:09.555663 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33387649-8b70-4e25-b664-4cb158212143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:09.576409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:09.577721 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-33387649-8b70-4e25-b664-4cb158212143 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:09.580321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-33387649-8b70-4e25-b664-4cb158212143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:09.626360 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51f81c-dd16-4ca0-92ed-bfb4734a7eab None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:09.669170 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4abd6-5c41-4975-a439-82060c30a2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:09.670282 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4abd6-5c41-4975-a439-82060c30a2b4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:09.670775 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4abd6-5c41-4975-a439-82060c30a2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:09.671316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4abd6-5c41-4975-a439-82060c30a2b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:09.675617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 351/707] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:52:09.676969 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33387649-8b70-4e25-b664-4cb158212143 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:09.692767 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4abd6-5c41-4975-a439-82060c30a2b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:09.694768 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:09.696950 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c4abd6-5c41-4975-a439-82060c30a2b4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:09.719453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-05c4abd6-5c41-4975-a439-82060c30a2b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:09.719453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:09.719453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:09.719453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:09.719453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:09.719453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:09.719453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:09.719453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:09.719453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:09.719453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:09.719453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:09.719453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:09.719453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:52:09.719453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:52:09.719453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:52:09.720494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:52:09.720494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:09.720494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:09.720494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:09.720494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:09.720494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:09.720494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:09.720494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:09.720494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:09.720494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:09.720494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:09.720494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:09.720494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:09.720494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:09.720494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:09.720494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:09.720494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:09.721409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:52:09.721409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:52:09.721409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:52:09.721409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:52:09.721409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:52:09.721409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:52:09.721409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:09.721409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:09.725379 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 357/708] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:09.732672 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4abd6-5c41-4975-a439-82060c30a2b4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:09.867029 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33387649-8b70-4e25-b664-4cb158212143 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:09.900875 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4abd6-5c41-4975-a439-82060c30a2b4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:09.926988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00ab9a-1322-4c67-96cd-5e3bed35e59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:09.927640 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00ab9a-1322-4c67-96cd-5e3bed35e59f None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:09.928192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00ab9a-1322-4c67-96cd-5e3bed35e59f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:09.928328 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00ab9a-1322-4c67-96cd-5e3bed35e59f None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:09.928939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00ab9a-1322-4c67-96cd-5e3bed35e59f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:09.930383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00ab9a-1322-4c67-96cd-5e3bed35e59f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:09.933918 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:09.941904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b596a-c4b3-4d4c-87ad-ab837e8aa367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:09.942814 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b596a-c4b3-4d4c-87ad-ab837e8aa367 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:09.943368 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b596a-c4b3-4d4c-87ad-ab837e8aa367 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:09.943917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b596a-c4b3-4d4c-87ad-ab837e8aa367 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:09.945001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b596a-c4b3-4d4c-87ad-ab837e8aa367 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:09.945396 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b596a-c4b3-4d4c-87ad-ab837e8aa367 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:09.947091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b596a-c4b3-4d4c-87ad-ab837e8aa367 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:09.957452 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 352/709] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 00:52:09 2022] POST /identity/v3/users => generated 254 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:52:09.959117 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773acc1-4472-4b94-8aa7-118e2cfb4689 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:09.959666 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773acc1-4472-4b94-8aa7-118e2cfb4689 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:09.960207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773acc1-4472-4b94-8aa7-118e2cfb4689 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:09.962080 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 353/710] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:09.967644 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 358/711] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:52:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:09.967790 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e53c1-bacf-4322-909d-26cd8cd59ddf None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:09.967790 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e53c1-bacf-4322-909d-26cd8cd59ddf None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:09.968078 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e53c1-bacf-4322-909d-26cd8cd59ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:09.975491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b596a-c4b3-4d4c-87ad-ab837e8aa367 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:09.990068 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-f43e53c1-bacf-4322-909d-26cd8cd59ddf None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:09.991321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f43e53c1-bacf-4322-909d-26cd8cd59ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:10.081410 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:10.092803 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 354/712] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:52:10.094592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e53c1-bacf-4322-909d-26cd8cd59ddf None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:10.140162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b596a-c4b3-4d4c-87ad-ab837e8aa367 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:10.200773 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af8907-7845-4c31-8657-27758aa0a0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:10.201687 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af8907-7845-4c31-8657-27758aa0a0e6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:10.202145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af8907-7845-4c31-8657-27758aa0a0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:10.202564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af8907-7845-4c31-8657-27758aa0a0e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:10.203653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af8907-7845-4c31-8657-27758aa0a0e6 None admin] RBAC: Adding query filter params (role.id=7a33c2ceb92a478f81753573f8ece1bd, scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3, user.id=e52776d9b4894af9b85a24136a8255e5) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:10.204134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af8907-7845-4c31-8657-27758aa0a0e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:10.205702 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af8907-7845-4c31-8657-27758aa0a0e6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:10.214641 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 359/713] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:52:09 2022] GET /identity/v3/role_assignments?user.id=e52776d9b4894af9b85a24136a8255e5&scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3&role.id=7a33c2ceb92a478f81753573f8ece1bd => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:10.219672 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e53c1-bacf-4322-909d-26cd8cd59ddf None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:10.258788 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2230e-6b0f-480c-a995-c52660f39cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:10.259459 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2230e-6b0f-480c-a995-c52660f39cb2 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:10.259758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2230e-6b0f-480c-a995-c52660f39cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:10.260029 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2230e-6b0f-480c-a995-c52660f39cb2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:10.260638 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2230e-6b0f-480c-a995-c52660f39cb2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:10.262087 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2230e-6b0f-480c-a995-c52660f39cb2 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:10.274901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-b6a2230e-6b0f-480c-a995-c52660f39cb2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 00:52:10.274901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:10.274901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:52:10.274901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:52:10.274901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:52:10.274901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:52:10.274901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:52:10.274901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:10.274901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 00:52:10.274901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:10.274901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:10.274901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:10.274901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:10.274901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:10.274901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:10.274901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:10.274901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:10.276517 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:10.276517 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:10.276517 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:10.276517 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:10.276517 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:10.276517 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:10.276517 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:52:10.276517 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:52:10.276517 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:52:10.276517 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:52:10.276517 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:10.276517 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:10.276517 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:10.276517 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:10.276517 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:10.276517 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:10.277365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:10.277365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:10.277365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:10.277365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:10.277365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:10.277365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:10.277365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:10.277365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:10.277365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:10.277365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:10.277365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:52:10.277365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:10.277365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 00:52:10.277365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:10.279660 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 355/714] 10.222.0.42 () {56 vars in 1129 bytes} [Wed Sep 14 00:52:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:10.282798 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af8907-7845-4c31-8657-27758aa0a0e6 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:10.349762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:10.404151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af8907-7845-4c31-8657-27758aa0a0e6 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:10.415501 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2230e-6b0f-480c-a995-c52660f39cb2 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:10.452233 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec275d13-f00f-48d2-ad6e-f9a4c28822f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:10.452857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec275d13-f00f-48d2-ad6e-f9a4c28822f8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:10.453715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec275d13-f00f-48d2-ad6e-f9a4c28822f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:10.454798 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec275d13-f00f-48d2-ad6e-f9a4c28822f8 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:10.455900 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec275d13-f00f-48d2-ad6e-f9a4c28822f8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:10.456352 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec275d13-f00f-48d2-ad6e-f9a4c28822f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:10.457978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec275d13-f00f-48d2-ad6e-f9a4c28822f8 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:10.464477 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 360/715] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Sep 14 00:52:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:10.475269 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec275d13-f00f-48d2-ad6e-f9a4c28822f8 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:10.507280 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:10.550714 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2230e-6b0f-480c-a995-c52660f39cb2 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:10.590057 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b169b112-9a00-44e8-b9b0-a15d16721ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:10.591081 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169b112-9a00-44e8-b9b0-a15d16721ebc None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:10.591602 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169b112-9a00-44e8-b9b0-a15d16721ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:10.592373 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169b112-9a00-44e8-b9b0-a15d16721ebc None admin] PATH_INFO: `/v3/services/network` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:10.593475 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b169b112-9a00-44e8-b9b0-a15d16721ebc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:10.595519 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b169b112-9a00-44e8-b9b0-a15d16721ebc None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:10.596871 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec275d13-f00f-48d2-ad6e-f9a4c28822f8 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:10.605356 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-b169b112-9a00-44e8-b9b0-a15d16721ebc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 00:52:10.605356 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:10.605356 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:52:10.605356 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:52:10.605356 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:52:10.605356 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:52:10.605356 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:52:10.605356 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:10.605356 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 00:52:10.605356 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:10.605356 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:10.605356 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:10.605356 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:10.605356 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:10.605356 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:10.605356 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:10.605356 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:10.606358 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:10.606358 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:10.606358 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:10.606358 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:10.606358 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:10.606358 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:10.606358 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:52:10.606358 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:52:10.606358 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:52:10.606358 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:52:10.606358 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:10.606358 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:10.606358 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:10.606358 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:10.606358 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:10.606358 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:10.607196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:10.607196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:10.607196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:10.607196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:10.607196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:10.607196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:10.607196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:10.607196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:10.607196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:10.607196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:10.607196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:52:10.607196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:10.607196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 00:52:10.607196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:10.610659 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 356/716] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:10 2022] GET /identity/v3/services/network => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:52:10.623615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b169b112-9a00-44e8-b9b0-a15d16721ebc None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:10.623807 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:10.646172 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d3c3ed-9f4c-4a20-86cd-ae034a4a8063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:10.647041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3c3ed-9f4c-4a20-86cd-ae034a4a8063 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:10.647491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3c3ed-9f4c-4a20-86cd-ae034a4a8063 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:10.647962 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3c3ed-9f4c-4a20-86cd-ae034a4a8063 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:10.648748 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d3c3ed-9f4c-4a20-86cd-ae034a4a8063 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:10.649189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d3c3ed-9f4c-4a20-86cd-ae034a4a8063 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:10.650712 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d3c3ed-9f4c-4a20-86cd-ae034a4a8063 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:10.657490 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 361/717] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Sep 14 00:52:10 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:10.754594 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b169b112-9a00-44e8-b9b0-a15d16721ebc None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:10.787575 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:10.806371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd43ad-e1cb-4994-bb7b-6362f1d76ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:10.807004 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd43ad-e1cb-4994-bb7b-6362f1d76ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:10.807284 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd43ad-e1cb-4994-bb7b-6362f1d76ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:10.807603 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd43ad-e1cb-4994-bb7b-6362f1d76ba7 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:10.808222 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd43ad-e1cb-4994-bb7b-6362f1d76ba7 None admin] RBAC: Adding query filter params (name=network) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:10.808523 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd43ad-e1cb-4994-bb7b-6362f1d76ba7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:10.809930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd43ad-e1cb-4994-bb7b-6362f1d76ba7 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:10.820239 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 357/718] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 00:52:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:10.826235 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d3c3ed-9f4c-4a20-86cd-ae034a4a8063 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:10.951506 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d3c3ed-9f4c-4a20-86cd-ae034a4a8063 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:10.988840 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322f95f-5783-4f5c-9e6a-679b0b26738e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:10.989457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322f95f-5783-4f5c-9e6a-679b0b26738e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:10.989736 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322f95f-5783-4f5c-9e6a-679b0b26738e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:10.990005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322f95f-5783-4f5c-9e6a-679b0b26738e None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:10.990633 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322f95f-5783-4f5c-9e6a-679b0b26738e None admin] RBAC: Adding query filter params (type=network) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:10.990935 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322f95f-5783-4f5c-9e6a-679b0b26738e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:10.992374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322f95f-5783-4f5c-9e6a-679b0b26738e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:10.997393 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 362/719] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 00:52:10 2022] GET /identity/v3/services?type=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:11.788081 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f0755-ec06-42b2-a165-1bdaaf551f1a None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:11.789102 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f0755-ec06-42b2-a165-1bdaaf551f1a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:11.789594 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f0755-ec06-42b2-a165-1bdaaf551f1a None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:11.791514 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 358/720] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:11.797781 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d9dc7-6afd-4ac1-b8b2-23e80cd2f4fe None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:11.798773 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d9dc7-6afd-4ac1-b8b2-23e80cd2f4fe None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:11.799259 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d9dc7-6afd-4ac1-b8b2-23e80cd2f4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:11.829075 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7c8d9dc7-6afd-4ac1-b8b2-23e80cd2f4fe None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:11.830620 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7c8d9dc7-6afd-4ac1-b8b2-23e80cd2f4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:11.886027 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:11.919400 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 363/721] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:52:12.430034 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f0755-ec06-42b2-a165-1bdaaf551f1a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:12.577189 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f0755-ec06-42b2-a165-1bdaaf551f1a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:12.621283 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721c11b-6d63-4a29-940f-cc36cca627dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:12.621927 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721c11b-6d63-4a29-940f-cc36cca627dd None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:12.622208 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721c11b-6d63-4a29-940f-cc36cca627dd None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:12.622483 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721c11b-6d63-4a29-940f-cc36cca627dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:12.644443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8721c11b-6d63-4a29-940f-cc36cca627dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:52:12.649734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-8721c11b-6d63-4a29-940f-cc36cca627dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:52:12.649734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:12.649734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:12.649734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:12.649734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:12.649734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:12.649734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:12.649734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:12.649734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:12.649734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:12.649734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:12.649734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:12.649734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:52:12.649734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:52:12.649734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:52:12.649734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:52:12.650687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:52:12.650687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:52:12.650687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:52:12.650687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:52:12.650687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:12.650687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:12.650687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:12.650687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:12.650687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:12.650687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:12.650687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:12.650687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:12.650687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:12.650687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:12.650687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:12.650687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:12.652188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:12.652188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:12.652188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:12.652188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:12.652188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:52:12.652188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:52:12.652188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:52:12.652188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:52:12.652188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:52:12.652188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:52:12.652188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:52:12.652188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:12.652188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 359/722] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:52:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:12.657018 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d9dc7-6afd-4ac1-b8b2-23e80cd2f4fe None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:12.811714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d9dc7-6afd-4ac1-b8b2-23e80cd2f4fe None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:12.862319 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7f14a-3bc0-441b-b41e-d76192cf45e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:12.862946 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7f14a-3bc0-441b-b41e-d76192cf45e0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:12.863229 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7f14a-3bc0-441b-b41e-d76192cf45e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:12.863506 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7f14a-3bc0-441b-b41e-d76192cf45e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:12.864212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7f14a-3bc0-441b-b41e-d76192cf45e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:12.864520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7f14a-3bc0-441b-b41e-d76192cf45e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:12.865916 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7f14a-3bc0-441b-b41e-d76192cf45e0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:12.879442 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 364/723] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:52:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:12.884974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721c11b-6d63-4a29-940f-cc36cca627dd None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:12.973229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:12.973801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacca7e-afca-4363-9cc5-2221ff682194 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:12.974093 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacca7e-afca-4363-9cc5-2221ff682194 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:12.974372 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aacca7e-afca-4363-9cc5-2221ff682194 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:12.975743 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 365/724] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:52:12.982333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f7386-3109-4df5-980c-4a2f505fa4d7 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:12.982626 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f7386-3109-4df5-980c-4a2f505fa4d7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:12.982913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f7386-3109-4df5-980c-4a2f505fa4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:13.008831 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-827f7386-3109-4df5-980c-4a2f505fa4d7 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:13.010484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-827f7386-3109-4df5-980c-4a2f505fa4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:13.042253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721c11b-6d63-4a29-940f-cc36cca627dd None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:13.049046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:13.081772 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc27eb-7f11-4f52-ad85-19643038cfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:13.082426 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc27eb-7f11-4f52-ad85-19643038cfc7 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:13.082710 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc27eb-7f11-4f52-ad85-19643038cfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:13.082995 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc27eb-7f11-4f52-ad85-19643038cfc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:13.095537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc27eb-7f11-4f52-ad85-19643038cfc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:13.096641 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 366/725] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:52:13.097061 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc27eb-7f11-4f52-ad85-19643038cfc7 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:13.098576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3807b2-3138-4c42-b29e-199481064dae None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:13.099061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3807b2-3138-4c42-b29e-199481064dae None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:13.099544 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3807b2-3138-4c42-b29e-199481064dae None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:13.105202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 367/726] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:13.107100 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ed39d-0fa2-4cb7-9815-6fb3a261b8d2 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:13.107652 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ed39d-0fa2-4cb7-9815-6fb3a261b8d2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:13.108225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ed39d-0fa2-4cb7-9815-6fb3a261b8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:13.127698 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-784ed39d-0fa2-4cb7-9815-6fb3a261b8d2 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:13.128316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-0cdc27eb-7f11-4f52-ad85-19643038cfc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:13.128316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:13.128316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:13.128316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:13.128316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:13.128316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:13.128316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:13.128316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:13.128316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:13.128316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:13.128316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:13.128316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:13.128316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:13.128316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:13.128316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:13.128316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:13.129255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:13.129255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:13.129255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:13.129255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:13.129255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:13.129255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:13.129255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:13.129255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:13.129255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:13.129255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:13.129255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:13.129255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:13.129255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:13.129255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:13.129255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:13.129255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:13.129255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:13.130141 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:13.130141 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:13.130141 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:13.131783 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-784ed39d-0fa2-4cb7-9815-6fb3a261b8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:13.134361 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 360/727] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:13.136790 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc27eb-7f11-4f52-ad85-19643038cfc7 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:13.211709 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 368/728] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:52:13.213114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c461d-9a79-420c-8fa5-2d95498a348d None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:13.213467 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c461d-9a79-420c-8fa5-2d95498a348d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:13.213811 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c461d-9a79-420c-8fa5-2d95498a348d None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:13.219649 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 369/729] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:13.221887 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28dccd-9650-4a5f-8cc4-0037b8699e6d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:13.222240 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28dccd-9650-4a5f-8cc4-0037b8699e6d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:13.222589 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28dccd-9650-4a5f-8cc4-0037b8699e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:13.254219 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4b28dccd-9650-4a5f-8cc4-0037b8699e6d None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:13.256931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4b28dccd-9650-4a5f-8cc4-0037b8699e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:13.259112 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc27eb-7f11-4f52-ad85-19643038cfc7 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:13.263728 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:13.321042 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d6347-4c65-4824-9c16-3e74172be0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:13.322048 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d6347-4c65-4824-9c16-3e74172be0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:13.323282 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d6347-4c65-4824-9c16-3e74172be0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:13.323767 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d6347-4c65-4824-9c16-3e74172be0a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:13.329481 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d6347-4c65-4824-9c16-3e74172be0a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:13.329976 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d6347-4c65-4824-9c16-3e74172be0a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:13.331653 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d6347-4c65-4824-9c16-3e74172be0a6 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:13.350287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 361/730] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:13.356193 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d6347-4c65-4824-9c16-3e74172be0a6 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:13.383288 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 370/731] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 00:52:13.444623 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28dccd-9650-4a5f-8cc4-0037b8699e6d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:13.490378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d6347-4c65-4824-9c16-3e74172be0a6 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:13.527059 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9c925-164d-4d81-ab2d-2131f80337f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:13.527731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9c925-164d-4d81-ab2d-2131f80337f5 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:13.528016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9c925-164d-4d81-ab2d-2131f80337f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:13.528321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9c925-164d-4d81-ab2d-2131f80337f5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:13.531637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:13.536557 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff9c925-164d-4d81-ab2d-2131f80337f5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:13.538040 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff9c925-164d-4d81-ab2d-2131f80337f5 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:13.551039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-dff9c925-164d-4d81-ab2d-2131f80337f5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 00:52:13.551039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:13.551039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:13.551039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:13.551039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:13.551039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:13.551039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:13.551039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:13.551039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:13.551039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:13.551039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:13.551039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:13.551039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:52:13.551039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:52:13.551039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:52:13.551039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:52:13.552026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:13.552026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:13.552026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:52:13.552026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:13.552026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:52:13.552026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:13.552026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:13.552026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:13.552026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:13.552026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:13.552026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:13.552026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:13.552026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:13.552026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:13.552026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:13.552026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:13.552026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:13.552026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:13.553538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:13.553538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:13.553538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:52:13.553538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:52:13.553538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:52:13.553538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:52:13.553538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:52:13.553538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:52:13.553538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 00:52:13.553538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:13.553538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 362/732] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Sep 14 00:52:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:13.553538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9c925-164d-4d81-ab2d-2131f80337f5 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:13.583855 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:13.639897 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28dccd-9650-4a5f-8cc4-0037b8699e6d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:13.681661 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2465624-5006-413b-b4af-ea052e346e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:13.682445 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2465624-5006-413b-b4af-ea052e346e0e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:13.682809 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2465624-5006-413b-b4af-ea052e346e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:13.683176 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2465624-5006-413b-b4af-ea052e346e0e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:13.683456 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9c925-164d-4d81-ab2d-2131f80337f5 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:13.696583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2465624-5006-413b-b4af-ea052e346e0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:52:13.702903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-e2465624-5006-413b-b4af-ea052e346e0e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:52:13.702903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:13.702903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:13.702903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:13.702903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:13.702903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:13.702903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:13.702903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:13.702903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:13.702903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:13.702903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:13.702903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:13.702903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:52:13.702903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:52:13.702903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:52:13.702903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:52:13.704118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:52:13.704118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:52:13.704118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:52:13.704118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:52:13.704118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:13.704118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:13.704118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:13.704118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:13.704118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:13.704118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:13.704118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:13.704118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:13.704118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:13.704118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:13.704118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:13.704118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:13.705006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:13.705006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:13.705006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:13.705006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:13.705006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:52:13.705006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:52:13.705006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:52:13.705006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:52:13.705006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:52:13.705006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:52:13.705006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:52:13.705006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:13.707632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 371/733] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 00:52:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:13.708668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2465624-5006-413b-b4af-ea052e346e0e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:13.731168 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee9926-1ca3-4719-a293-32498b90a91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:13.732559 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee9926-1ca3-4719-a293-32498b90a91d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:13.733295 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee9926-1ca3-4719-a293-32498b90a91d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:13.733759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee9926-1ca3-4719-a293-32498b90a91d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:13.745675 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee9926-1ca3-4719-a293-32498b90a91d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:52:13.751453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-7cee9926-1ca3-4719-a293-32498b90a91d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:52:13.751453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:13.751453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:13.751453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:13.751453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:13.751453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:13.751453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:13.751453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:13.751453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:13.751453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:13.751453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:13.751453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:13.751453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:52:13.751453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:52:13.751453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:52:13.751453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:52:13.752429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:52:13.752429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:52:13.752429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:52:13.752429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:52:13.752429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:13.752429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:13.752429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:13.752429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:13.752429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:13.752429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:13.752429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:13.752429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:13.752429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:13.752429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:13.752429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:13.752429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:13.754585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:13.754585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:13.754585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:13.754585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:13.754585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:52:13.754585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:52:13.754585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:52:13.754585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:52:13.754585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:52:13.754585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:52:13.754585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:52:13.754585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:13.754585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 363/734] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:52:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:13.754585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee9926-1ca3-4719-a293-32498b90a91d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:13.818888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:13.834130 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2465624-5006-413b-b4af-ea052e346e0e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:13.843241 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:13.863137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee9926-1ca3-4719-a293-32498b90a91d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:13.870779 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcef741-a246-4f54-8da7-e635f805d2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:13.871394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcef741-a246-4f54-8da7-e635f805d2ef None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:13.871754 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcef741-a246-4f54-8da7-e635f805d2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:13.872032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcef741-a246-4f54-8da7-e635f805d2ef None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:13.872759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcef741-a246-4f54-8da7-e635f805d2ef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:13.873067 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcef741-a246-4f54-8da7-e635f805d2ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:13.874497 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcef741-a246-4f54-8da7-e635f805d2ef None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:13.898675 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 372/735] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Sep 14 00:52:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:13.898675 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcef741-a246-4f54-8da7-e635f805d2ef None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:13.902101 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2184a8-f79d-411a-b332-8e0a1b86b80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:13.902709 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2184a8-f79d-411a-b332-8e0a1b86b80c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:13.902989 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2184a8-f79d-411a-b332-8e0a1b86b80c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:13.903260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2184a8-f79d-411a-b332-8e0a1b86b80c None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:13.903948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2184a8-f79d-411a-b332-8e0a1b86b80c None admin] RBAC: Adding query filter params (name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:13.904250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2184a8-f79d-411a-b332-8e0a1b86b80c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:13.905611 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2184a8-f79d-411a-b332-8e0a1b86b80c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:13.919139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 364/736] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 00:52:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:13.920025 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2184a8-f79d-411a-b332-8e0a1b86b80c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:14.010907 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcef741-a246-4f54-8da7-e635f805d2ef None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:14.033900 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2184a8-f79d-411a-b332-8e0a1b86b80c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:14.051735 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabec513-20d2-4447-92bc-af5e7e393172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:14.052593 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabec513-20d2-4447-92bc-af5e7e393172 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:14.053273 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabec513-20d2-4447-92bc-af5e7e393172 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:14.053606 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabec513-20d2-4447-92bc-af5e7e393172 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:14.054263 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabec513-20d2-4447-92bc-af5e7e393172 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:14.055685 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabec513-20d2-4447-92bc-af5e7e393172 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:14.070687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b1075-b004-4db5-a44e-1fdb24da8f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:14.071307 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b1075-b004-4db5-a44e-1fdb24da8f92 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:14.071625 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b1075-b004-4db5-a44e-1fdb24da8f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:14.071905 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b1075-b004-4db5-a44e-1fdb24da8f92 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:14.072721 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73b1075-b004-4db5-a44e-1fdb24da8f92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:14.073206 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73b1075-b004-4db5-a44e-1fdb24da8f92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:14.074438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73b1075-b004-4db5-a44e-1fdb24da8f92 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:14.074677 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 373/737] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 00:52:13 2022] POST /identity/v3/services => generated 251 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:52:14.075494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabec513-20d2-4447-92bc-af5e7e393172 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:14.089055 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 365/738] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:52:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:14.089055 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b1075-b004-4db5-a44e-1fdb24da8f92 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:14.089342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:14.102230 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:14.208469 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b1075-b004-4db5-a44e-1fdb24da8f92 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:14.217006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabec513-20d2-4447-92bc-af5e7e393172 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:14.246547 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5d2d0-aaa5-491e-b72a-dcbf0fa6c2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:14.247197 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5d2d0-aaa5-491e-b72a-dcbf0fa6c2be None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:14.247475 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5d2d0-aaa5-491e-b72a-dcbf0fa6c2be None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:14.247785 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5d2d0-aaa5-491e-b72a-dcbf0fa6c2be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:14.257370 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b8710-814a-4c5a-997e-c9526c634987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:14.257836 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b8710-814a-4c5a-997e-c9526c634987 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:14.258123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b8710-814a-4c5a-997e-c9526c634987 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:14.258397 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b8710-814a-4c5a-997e-c9526c634987 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:14.260338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5d2d0-aaa5-491e-b72a-dcbf0fa6c2be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:14.261789 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5d2d0-aaa5-491e-b72a-dcbf0fa6c2be None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:14.271736 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b8710-814a-4c5a-997e-c9526c634987 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:14.273294 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b8710-814a-4c5a-997e-c9526c634987 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:14.278342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-a0d5d2d0-aaa5-491e-b72a-dcbf0fa6c2be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:14.278342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:14.278342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:14.278342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:14.278342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:14.278342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:14.278342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:14.278342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:14.278342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:14.278342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:14.278342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:14.278342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:14.278342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:14.278342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:14.278342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:14.278342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:14.279300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:14.279300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:14.279300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:14.279300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:14.279300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:14.279300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:14.279300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:14.279300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:14.279300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:14.279300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:14.279300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:14.279300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:14.279300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:14.279300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:14.279300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:14.279300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:14.279300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:14.280689 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:14.280689 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:14.280689 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:14.280689 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 366/739] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:14.281651 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e543e4-f94c-48d0-be1b-16d672dfb81f None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:14.281926 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e543e4-f94c-48d0-be1b-16d672dfb81f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:14.282613 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e543e4-f94c-48d0-be1b-16d672dfb81f None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:14.284136 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 367/740] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:14.285064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e543e4-f94c-48d0-be1b-16d672dfb81f None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:14.289017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-d58b8710-814a-4c5a-997e-c9526c634987 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:14.289017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:14.289017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:14.289017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:14.289017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:14.289017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:14.289017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:14.289017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:14.289017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:14.289017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:14.289017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:14.289017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:14.289017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:14.289017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:14.289017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:14.289017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:14.290339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:14.290339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:14.290339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:14.290339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:14.290339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:14.290339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:14.290339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:14.290339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:14.290339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:14.290339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:14.290339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:14.290339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:14.290339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:14.290339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:14.290339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:14.290339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:14.290339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:14.291686 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:14.291686 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:14.291686 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:14.291686 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 374/741] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:14.291686 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b8710-814a-4c5a-997e-c9526c634987 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:14.350812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:14.355640 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:14.397894 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e543e4-f94c-48d0-be1b-16d672dfb81f None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:14.404141 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b8710-814a-4c5a-997e-c9526c634987 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:14.433774 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a5afb-6ee1-4df3-a592-477f160252ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:14.434409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a5afb-6ee1-4df3-a592-477f160252ff None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:14.434851 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a5afb-6ee1-4df3-a592-477f160252ff None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:14.435129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a5afb-6ee1-4df3-a592-477f160252ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:14.441033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73546cae-1592-48a8-b9fc-39e4e9a6efff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:14.441640 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73546cae-1592-48a8-b9fc-39e4e9a6efff None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:14.441916 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73546cae-1592-48a8-b9fc-39e4e9a6efff None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:14.442187 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73546cae-1592-48a8-b9fc-39e4e9a6efff None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:14.442823 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73546cae-1592-48a8-b9fc-39e4e9a6efff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:14.443119 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73546cae-1592-48a8-b9fc-39e4e9a6efff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:14.444661 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73546cae-1592-48a8-b9fc-39e4e9a6efff None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:14.448098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8a5afb-6ee1-4df3-a592-477f160252ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:14.449547 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8a5afb-6ee1-4df3-a592-477f160252ff None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:14.461678 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 375/742] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:14.461678 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a038e-d6aa-4893-b22a-cac8102b0747 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:14.461678 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a038e-d6aa-4893-b22a-cac8102b0747 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:14.461678 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a038e-d6aa-4893-b22a-cac8102b0747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:14.472751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-0d8a5afb-6ee1-4df3-a592-477f160252ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:14.472751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:14.472751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:14.472751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:14.472751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:14.472751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:14.472751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:14.472751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:14.472751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:14.472751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:14.472751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:14.472751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:14.472751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:14.472751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:14.472751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:14.472751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:14.473765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:14.473765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:14.473765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:14.473765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:14.473765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:14.473765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:14.473765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:14.473765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:14.473765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:14.473765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:14.473765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:14.473765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:14.473765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:14.473765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:14.473765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:14.473765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:14.473765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:14.474676 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:14.474676 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:14.474676 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:14.475677 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 368/743] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:14.476731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a5afb-6ee1-4df3-a592-477f160252ff None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:14.479614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-073a038e-d6aa-4893-b22a-cac8102b0747 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:14.481034 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-073a038e-d6aa-4893-b22a-cac8102b0747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:14.554652 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 376/744] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:14 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 00:52:14.555621 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a038e-d6aa-4893-b22a-cac8102b0747 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:14.604195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a5afb-6ee1-4df3-a592-477f160252ff None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:14.613645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:14.621554 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:14.650564 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238257a6-6310-40cc-9c08-a242d3584c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:14.651423 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238257a6-6310-40cc-9c08-a242d3584c66 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:14.652397 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238257a6-6310-40cc-9c08-a242d3584c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:14.652883 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238257a6-6310-40cc-9c08-a242d3584c66 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:14.654546 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238257a6-6310-40cc-9c08-a242d3584c66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:14.655216 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238257a6-6310-40cc-9c08-a242d3584c66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:14.658211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238257a6-6310-40cc-9c08-a242d3584c66 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:14.675352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 369/745] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:14.676746 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238257a6-6310-40cc-9c08-a242d3584c66 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:14.681345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a038e-d6aa-4893-b22a-cac8102b0747 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:14.729178 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f3323-7022-4268-8e27-c02765c3fd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:14.729731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f3323-7022-4268-8e27-c02765c3fd31 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:14.730009 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f3323-7022-4268-8e27-c02765c3fd31 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:14.730281 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f3323-7022-4268-8e27-c02765c3fd31 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:14.740072 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f3323-7022-4268-8e27-c02765c3fd31 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:14.741909 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f3323-7022-4268-8e27-c02765c3fd31 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:14.758136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-5c7f3323-7022-4268-8e27-c02765c3fd31 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 00:52:14.758136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:14.758136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:14.758136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:14.758136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:14.758136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:14.758136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:14.758136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:14.758136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:14.758136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:14.758136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:14.758136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:14.758136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:52:14.758136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:52:14.758136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:52:14.758136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:52:14.759402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:14.759402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:14.759402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:52:14.759402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:14.759402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:52:14.759402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:14.759402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:14.759402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:14.759402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:14.759402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:14.759402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:14.759402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:14.759402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:14.759402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:14.759402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:14.759402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:14.759402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:14.759402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:14.760568 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:14.760568 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:14.760568 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:52:14.760568 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:52:14.760568 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:52:14.760568 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:52:14.760568 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:52:14.760568 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:52:14.760568 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 00:52:14.760568 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:14.763696 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 377/746] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Sep 14 00:52:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:14.764620 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f3323-7022-4268-8e27-c02765c3fd31 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:14.804806 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238257a6-6310-40cc-9c08-a242d3584c66 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:14.841670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce8b6c-c5b8-4d19-8f37-2b9c9f01142b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:14.842400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce8b6c-c5b8-4d19-8f37-2b9c9f01142b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:14.842750 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce8b6c-c5b8-4d19-8f37-2b9c9f01142b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:14.843112 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce8b6c-c5b8-4d19-8f37-2b9c9f01142b None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:14.843978 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce8b6c-c5b8-4d19-8f37-2b9c9f01142b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:14.844675 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce8b6c-c5b8-4d19-8f37-2b9c9f01142b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:14.846139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce8b6c-c5b8-4d19-8f37-2b9c9f01142b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:14.857929 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 370/747] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:14.858812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce8b6c-c5b8-4d19-8f37-2b9c9f01142b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:14.892287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:14.894678 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f3323-7022-4268-8e27-c02765c3fd31 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:14.898941 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:14.940038 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d638fd-558a-44cb-8dc2-28002b2849fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:14.940676 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d638fd-558a-44cb-8dc2-28002b2849fb None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:14.940962 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d638fd-558a-44cb-8dc2-28002b2849fb None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:14.941588 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d638fd-558a-44cb-8dc2-28002b2849fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:14.950308 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d638fd-558a-44cb-8dc2-28002b2849fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:14.951786 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d638fd-558a-44cb-8dc2-28002b2849fb None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:14.963137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-b5d638fd-558a-44cb-8dc2-28002b2849fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:14.963137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:14.963137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:14.963137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:14.963137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:14.963137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:14.963137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:14.963137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:14.963137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:14.963137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:14.963137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:14.963137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:14.963137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:52:14.963137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:52:14.963137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:52:14.964071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:52:14.964071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:14.964071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:14.964071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:14.964071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:14.964071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:14.964071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:14.964071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:14.964071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:14.964071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:14.964071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:14.964071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:14.964071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:14.964071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:14.964071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:14.964071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:14.964071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:14.964958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:52:14.964958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:52:14.964958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:52:14.964958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:52:14.964958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:52:14.964958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:52:14.964958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:14.964958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:14.965379 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce8b6c-c5b8-4d19-8f37-2b9c9f01142b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:14.968912 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 378/748] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:14.969710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d638fd-558a-44cb-8dc2-28002b2849fb None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:15.005048 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d282dc-7c44-44a2-a6df-91502d300cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:15.005670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d282dc-7c44-44a2-a6df-91502d300cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:15.005953 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d282dc-7c44-44a2-a6df-91502d300cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:15.006221 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d282dc-7c44-44a2-a6df-91502d300cb0 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:15.006949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d282dc-7c44-44a2-a6df-91502d300cb0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:15.007253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d282dc-7c44-44a2-a6df-91502d300cb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:15.008740 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d282dc-7c44-44a2-a6df-91502d300cb0 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:15.031601 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 371/749] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Sep 14 00:52:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:15.031601 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d282dc-7c44-44a2-a6df-91502d300cb0 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:15.086827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d638fd-558a-44cb-8dc2-28002b2849fb None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:15.119933 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:15.124002 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189bd6f9-ae38-42a8-834f-a60883b5b741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:15.124639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bd6f9-ae38-42a8-834f-a60883b5b741 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:15.125465 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bd6f9-ae38-42a8-834f-a60883b5b741 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:15.126122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bd6f9-ae38-42a8-834f-a60883b5b741 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:15.133662 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189bd6f9-ae38-42a8-834f-a60883b5b741 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:15.134037 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d282dc-7c44-44a2-a6df-91502d300cb0 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:15.135101 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189bd6f9-ae38-42a8-834f-a60883b5b741 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:15.147335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-189bd6f9-ae38-42a8-834f-a60883b5b741 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 00:52:15.147335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:15.147335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:15.147335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:15.147335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:15.147335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:15.147335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:15.147335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:15.147335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:15.147335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:15.147335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:15.147335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:15.147335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:52:15.147335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:52:15.147335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:52:15.147335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:52:15.148342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:15.148342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:15.148342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:52:15.148342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:15.148342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:52:15.148342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:15.148342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:15.148342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:15.148342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:15.148342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:15.148342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:15.148342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:15.148342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:15.148342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:15.148342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:15.148342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:15.148342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:15.148342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:15.149300 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:15.149300 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:15.149300 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:52:15.149300 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:52:15.149300 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:52:15.149300 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:52:15.149300 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:52:15.149300 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:52:15.149300 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 00:52:15.149300 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:15.149300 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 379/750] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:52:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:15.149300 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189bd6f9-ae38-42a8-834f-a60883b5b741 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:15.176937 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a342e-d660-43f7-b9b1-5af3d9301062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:15.177549 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a342e-d660-43f7-b9b1-5af3d9301062 None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:15.177836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a342e-d660-43f7-b9b1-5af3d9301062 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:15.178108 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a342e-d660-43f7-b9b1-5af3d9301062 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:15.178753 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a342e-d660-43f7-b9b1-5af3d9301062 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:15.180170 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a342e-d660-43f7-b9b1-5af3d9301062 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:15.180590 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:15.199598 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 372/751] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 00:52:15 2022] POST /identity/v3/services => generated 240 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:52:15.199598 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a342e-d660-43f7-b9b1-5af3d9301062 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:15.280846 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189bd6f9-ae38-42a8-834f-a60883b5b741 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:15.338086 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ef991-f987-4d59-9436-1f1811e9ba30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:15.338723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ef991-f987-4d59-9436-1f1811e9ba30 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:15.339012 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ef991-f987-4d59-9436-1f1811e9ba30 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:15.339486 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ef991-f987-4d59-9436-1f1811e9ba30 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:15.340706 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ef991-f987-4d59-9436-1f1811e9ba30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:15.341034 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ef991-f987-4d59-9436-1f1811e9ba30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:15.342825 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ef991-f987-4d59-9436-1f1811e9ba30 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:15.351815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a342e-d660-43f7-b9b1-5af3d9301062 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:15.360757 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 380/752] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:52:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:15.361729 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ef991-f987-4d59-9436-1f1811e9ba30 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:15.411607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c8f1f-64f3-43ec-9248-350fbc146367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:15.412240 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c8f1f-64f3-43ec-9248-350fbc146367 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:15.412526 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c8f1f-64f3-43ec-9248-350fbc146367 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:15.412801 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c8f1f-64f3-43ec-9248-350fbc146367 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:15.438138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:15.465689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:15.495571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ef991-f987-4d59-9436-1f1811e9ba30 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:15.508433 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c8f1f-64f3-43ec-9248-350fbc146367 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:15.510484 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9c8f1f-64f3-43ec-9248-350fbc146367 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:15.537785 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a9592-d821-4e80-971e-a7a85312dd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:15.538609 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a9592-d821-4e80-971e-a7a85312dd46 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:15.538892 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a9592-d821-4e80-971e-a7a85312dd46 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:15.539169 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a9592-d821-4e80-971e-a7a85312dd46 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:15.540142 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a9592-d821-4e80-971e-a7a85312dd46 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:15.540466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a9592-d821-4e80-971e-a7a85312dd46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:15.541913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a9592-d821-4e80-971e-a7a85312dd46 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:15.565558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 381/753] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Sep 14 00:52:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:15.566680 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a9592-d821-4e80-971e-a7a85312dd46 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:15.623153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 373/754] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:52:15 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 00:52:15.624062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c8f1f-64f3-43ec-9248-350fbc146367 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:15.680174 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:15.690963 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a9592-d821-4e80-971e-a7a85312dd46 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:15.737475 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64921526-1f12-44ba-8b99-87012785f20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:15.739115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64921526-1f12-44ba-8b99-87012785f20e None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:15.739586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64921526-1f12-44ba-8b99-87012785f20e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:15.740016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64921526-1f12-44ba-8b99-87012785f20e None admin] PATH_INFO: `/v3/projects/fda69e8c2d5e466dbc1244b2f76296a3/users/5e0fe2e84f6748c992c9beedf2f5d49b/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:15.740817 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9c8f1f-64f3-43ec-9248-350fbc146367 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:15.747764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64921526-1f12-44ba-8b99-87012785f20e None admin] RBAC: Authorizing `identity:create_grant(project_id=fda69e8c2d5e466dbc1244b2f76296a3, user_id=5e0fe2e84f6748c992c9beedf2f5d49b, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:15.747764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64921526-1f12-44ba-8b99-87012785f20e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:15.755229 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:15.772028 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 382/755] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 00:52:15 2022] PUT /identity/v3/projects/fda69e8c2d5e466dbc1244b2f76296a3/users/5e0fe2e84f6748c992c9beedf2f5d49b/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:52:15.773450 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64921526-1f12-44ba-8b99-87012785f20e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:15.789337 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8fde7a-c68b-4663-9c34-cabb1f5ff30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:15.790385 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fde7a-c68b-4663-9c34-cabb1f5ff30e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:15.790884 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fde7a-c68b-4663-9c34-cabb1f5ff30e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:15.791360 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fde7a-c68b-4663-9c34-cabb1f5ff30e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:15.941144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8fde7a-c68b-4663-9c34-cabb1f5ff30e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:15.942658 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8fde7a-c68b-4663-9c34-cabb1f5ff30e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:15.982425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64921526-1f12-44ba-8b99-87012785f20e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:16.024429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:16.030727 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791d928-6b7f-4f60-9f2e-af4c3c746110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:16.031667 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791d928-6b7f-4f60-9f2e-af4c3c746110 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:16.031971 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791d928-6b7f-4f60-9f2e-af4c3c746110 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:16.032453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791d928-6b7f-4f60-9f2e-af4c3c746110 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:16.043993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791d928-6b7f-4f60-9f2e-af4c3c746110 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:16.045389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791d928-6b7f-4f60-9f2e-af4c3c746110 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:16.065630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-2791d928-6b7f-4f60-9f2e-af4c3c746110 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:16.065630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:16.065630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:16.065630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:16.065630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:16.065630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:16.065630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:16.065630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:16.065630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:16.065630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:16.065630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:16.065630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:16.065630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:16.065630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:16.065630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:16.065630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:16.066659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:16.066659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:16.066659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:16.066659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:16.066659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:16.066659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:16.066659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:16.066659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:16.066659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:16.066659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:16.066659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:16.066659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:16.066659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:16.066659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:16.066659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:16.066659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:16.066659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:16.067590 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:16.067590 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:16.067590 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:16.067590 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 383/756] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:16.076353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791d928-6b7f-4f60-9f2e-af4c3c746110 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:16.090604 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 374/757] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:52:15 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 00:52:16.099604 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8fde7a-c68b-4663-9c34-cabb1f5ff30e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:16.189701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:16.275481 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791d928-6b7f-4f60-9f2e-af4c3c746110 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:16.279731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8fde7a-c68b-4663-9c34-cabb1f5ff30e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:16.353299 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010daf8b-cbe9-46cc-bce1-894f4175c49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:16.354536 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1e71e-31d9-4c13-b9f0-3ffbd9967706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:16.355193 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1e71e-31d9-4c13-b9f0-3ffbd9967706 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:16.355475 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1e71e-31d9-4c13-b9f0-3ffbd9967706 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:16.355776 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1e71e-31d9-4c13-b9f0-3ffbd9967706 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:16.356825 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010daf8b-cbe9-46cc-bce1-894f4175c49e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:16.357100 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010daf8b-cbe9-46cc-bce1-894f4175c49e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:16.357372 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010daf8b-cbe9-46cc-bce1-894f4175c49e None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:16.358026 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010daf8b-cbe9-46cc-bce1-894f4175c49e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:16.358335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010daf8b-cbe9-46cc-bce1-894f4175c49e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:16.361103 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010daf8b-cbe9-46cc-bce1-894f4175c49e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:16.379131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1e71e-31d9-4c13-b9f0-3ffbd9967706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:16.380641 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1e71e-31d9-4c13-b9f0-3ffbd9967706 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:16.385114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 384/758] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:16.392798 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010daf8b-cbe9-46cc-bce1-894f4175c49e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:16.401153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:16.409608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-e3b1e71e-31d9-4c13-b9f0-3ffbd9967706 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:16.409608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:16.409608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:16.409608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:16.409608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:16.409608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:16.409608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:16.409608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:16.409608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:16.409608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:16.409608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:16.409608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:16.409608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:16.409608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:16.409608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:16.409608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:16.411796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:16.411796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:16.411796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:16.411796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:16.411796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:16.411796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:16.411796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:16.411796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:16.411796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:16.411796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:16.411796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:16.411796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:16.411796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:16.411796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:16.411796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:16.411796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:16.411796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:16.413892 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:16.413892 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:16.413892 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:16.413892 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 375/759] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:16.413892 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae295a-dd44-499d-8639-3062660ddd2b None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:16.413892 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae295a-dd44-499d-8639-3062660ddd2b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:16.413892 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae295a-dd44-499d-8639-3062660ddd2b None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:16.414789 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 376/760] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:16.420545 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae295a-dd44-499d-8639-3062660ddd2b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:16.517801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010daf8b-cbe9-46cc-bce1-894f4175c49e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:16.531960 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:16.557549 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae295a-dd44-499d-8639-3062660ddd2b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:16.563080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986479fd-ac8a-441b-8773-144a25cd4cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:16.563766 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986479fd-ac8a-441b-8773-144a25cd4cca None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:16.564053 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986479fd-ac8a-441b-8773-144a25cd4cca None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:16.564338 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986479fd-ac8a-441b-8773-144a25cd4cca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:16.572741 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986479fd-ac8a-441b-8773-144a25cd4cca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:16.574609 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986479fd-ac8a-441b-8773-144a25cd4cca None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:16.589079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-986479fd-ac8a-441b-8773-144a25cd4cca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:16.589079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:16.589079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:16.589079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:16.589079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:16.589079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:16.589079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:16.589079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:16.589079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:16.589079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:16.589079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:16.589079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:16.589079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:52:16.589079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:52:16.589079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:52:16.591186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:52:16.591186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:16.591186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:16.591186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:16.591186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:16.591186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:16.591186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:16.591186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:16.591186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:16.591186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:16.591186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:16.591186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:16.591186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:16.591186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:16.591186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:16.591186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:16.591186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:16.593543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:52:16.593543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:52:16.593543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:52:16.593543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:52:16.593543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:52:16.593543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:52:16.593543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:16.593543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:16.593543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 385/761] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:16.593543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5cd86d-29ad-4885-819a-cd638e840f5e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:16.593543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5cd86d-29ad-4885-819a-cd638e840f5e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:16.595398 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5cd86d-29ad-4885-819a-cd638e840f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:16.598448 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe9d4d-4131-4494-9bfc-40c92942f4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:16.598448 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe9d4d-4131-4494-9bfc-40c92942f4f3 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:16.598448 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe9d4d-4131-4494-9bfc-40c92942f4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:16.598448 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe9d4d-4131-4494-9bfc-40c92942f4f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:16.598802 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe9d4d-4131-4494-9bfc-40c92942f4f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:16.598802 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe9d4d-4131-4494-9bfc-40c92942f4f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:16.599528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe9d4d-4131-4494-9bfc-40c92942f4f3 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:16.612976 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 377/762] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:16.613944 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe9d4d-4131-4494-9bfc-40c92942f4f3 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:16.616824 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-5c5cd86d-29ad-4885-819a-cd638e840f5e None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:16.619291 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5c5cd86d-29ad-4885-819a-cd638e840f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:16.680815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:16.718958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 386/763] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:16.720310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5cd86d-29ad-4885-819a-cd638e840f5e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:16.735946 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe9d4d-4131-4494-9bfc-40c92942f4f3 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:16.821053 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0a916-895d-44e5-ac85-442470db4f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:16.821853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0a916-895d-44e5-ac85-442470db4f70 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:16.822197 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0a916-895d-44e5-ac85-442470db4f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:16.822539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0a916-895d-44e5-ac85-442470db4f70 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:16.823587 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0a916-895d-44e5-ac85-442470db4f70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:16.823738 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0a916-895d-44e5-ac85-442470db4f70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:16.825219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0a916-895d-44e5-ac85-442470db4f70 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:16.835266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:16.839472 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5cd86d-29ad-4885-819a-cd638e840f5e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:16.857983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 378/764] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:52:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:16.864352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0a916-895d-44e5-ac85-442470db4f70 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:16.881853 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2eec73-99c1-4b9d-9928-789f9b722686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:16.882743 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2eec73-99c1-4b9d-9928-789f9b722686 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:16.883209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2eec73-99c1-4b9d-9928-789f9b722686 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:16.883994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2eec73-99c1-4b9d-9928-789f9b722686 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:16.892035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2eec73-99c1-4b9d-9928-789f9b722686 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:16.894236 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2eec73-99c1-4b9d-9928-789f9b722686 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:16.907389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-9f2eec73-99c1-4b9d-9928-789f9b722686 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:16.907389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:16.907389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:16.907389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:16.907389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:16.907389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:16.907389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:16.907389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:16.907389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:16.907389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:16.907389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:16.907389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:16.907389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:52:16.907389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:52:16.907389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:52:16.908333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:52:16.908333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:16.908333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:16.908333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:16.908333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:16.908333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:16.908333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:16.908333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:16.908333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:16.908333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:16.908333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:16.908333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:16.908333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:16.908333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:16.908333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:16.908333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:16.908333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:16.909285 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:52:16.909285 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:52:16.909285 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:52:16.909285 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:52:16.909285 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:52:16.909285 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:52:16.909285 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:16.909285 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:16.912167 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 387/765] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:16.920608 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2eec73-99c1-4b9d-9928-789f9b722686 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:17.049065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2eec73-99c1-4b9d-9928-789f9b722686 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:17.079687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0a916-895d-44e5-ac85-442470db4f70 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:17.095004 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:17.113668 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884aca1c-7930-4c2b-86c9-a4f35cc80da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:17.114282 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884aca1c-7930-4c2b-86c9-a4f35cc80da8 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:17.114612 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884aca1c-7930-4c2b-86c9-a4f35cc80da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:17.114834 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884aca1c-7930-4c2b-86c9-a4f35cc80da8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:17.115548 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884aca1c-7930-4c2b-86c9-a4f35cc80da8 None admin] RBAC: Adding query filter params (role.id=7a33c2ceb92a478f81753573f8ece1bd, scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3, user.id=ceccc909529042539dd2d5015058d8f7) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:17.115873 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884aca1c-7930-4c2b-86c9-a4f35cc80da8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:17.117317 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884aca1c-7930-4c2b-86c9-a4f35cc80da8 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:17.120255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c421e2-ad01-4af4-aa44-2e144db74064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:17.121351 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c421e2-ad01-4af4-aa44-2e144db74064 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:17.121632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c421e2-ad01-4af4-aa44-2e144db74064 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:17.122050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c421e2-ad01-4af4-aa44-2e144db74064 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:17.122750 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c421e2-ad01-4af4-aa44-2e144db74064 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:17.123056 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c421e2-ad01-4af4-aa44-2e144db74064 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:17.124637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c421e2-ad01-4af4-aa44-2e144db74064 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:17.126757 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 379/766] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:52:16 2022] GET /identity/v3/role_assignments?user.id=ceccc909529042539dd2d5015058d8f7&scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3&role.id=7a33c2ceb92a478f81753573f8ece1bd => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:17.127976 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884aca1c-7930-4c2b-86c9-a4f35cc80da8 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:17.140915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 388/767] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:52:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:17.146676 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c421e2-ad01-4af4-aa44-2e144db74064 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:17.172649 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:17.290178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884aca1c-7930-4c2b-86c9-a4f35cc80da8 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:17.297963 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c421e2-ad01-4af4-aa44-2e144db74064 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:17.338457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff32f7-e38f-46fc-8fcf-ce3bebb51bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:17.339094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff32f7-e38f-46fc-8fcf-ce3bebb51bdb None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:17.339387 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff32f7-e38f-46fc-8fcf-ce3bebb51bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:17.339691 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff32f7-e38f-46fc-8fcf-ce3bebb51bdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:17.340442 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff32f7-e38f-46fc-8fcf-ce3bebb51bdb None admin] RBAC: Adding query filter params (role.id=ada268ea63354a0297070369dfe3c6cc, scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3, user.id=e52776d9b4894af9b85a24136a8255e5) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:17.340757 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff32f7-e38f-46fc-8fcf-ce3bebb51bdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:17.342758 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff32f7-e38f-46fc-8fcf-ce3bebb51bdb None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:17.349055 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 389/768] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:52:17 2022] GET /identity/v3/role_assignments?user.id=e52776d9b4894af9b85a24136a8255e5&scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3&role.id=ada268ea63354a0297070369dfe3c6cc => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:17.369944 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c961525-5780-461b-956b-cf087df93492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:17.370528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c961525-5780-461b-956b-cf087df93492 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:17.370812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c961525-5780-461b-956b-cf087df93492 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:17.371094 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c961525-5780-461b-956b-cf087df93492 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:17.371770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c961525-5780-461b-956b-cf087df93492 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:17.373203 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c961525-5780-461b-956b-cf087df93492 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:17.383986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-3c961525-5780-461b-956b-cf087df93492 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 00:52:17.383986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:17.383986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:52:17.383986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:52:17.383986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:52:17.383986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:52:17.383986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:52:17.383986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:17.383986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 00:52:17.383986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:17.383986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:17.383986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:17.383986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:17.383986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:17.383986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:17.383986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:17.383986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:17.385290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:17.385290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:17.385290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:17.385290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:17.385290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:17.385290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:17.385290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:52:17.385290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:52:17.385290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:52:17.385290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:52:17.385290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:17.385290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:17.385290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:17.385290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:17.385290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:17.385290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:17.386326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:17.386326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:17.386326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:17.386326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:17.386326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:17.386326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:17.386326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:17.386326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:17.386326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:17.386326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:17.386326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:52:17.386326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:17.386326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 00:52:17.386326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:17.386326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 380/769] 10.222.0.42 () {56 vars in 1129 bytes} [Wed Sep 14 00:52:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:17.394005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff32f7-e38f-46fc-8fcf-ce3bebb51bdb None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:17.511972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:17.533800 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff32f7-e38f-46fc-8fcf-ce3bebb51bdb None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:17.580288 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02faf2-fcbc-4474-bd89-1f3c5654494a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:17.580913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02faf2-fcbc-4474-bd89-1f3c5654494a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:17.581200 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02faf2-fcbc-4474-bd89-1f3c5654494a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:17.581477 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02faf2-fcbc-4474-bd89-1f3c5654494a None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:17.582117 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02faf2-fcbc-4474-bd89-1f3c5654494a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:17.582425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02faf2-fcbc-4474-bd89-1f3c5654494a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:17.589087 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02faf2-fcbc-4474-bd89-1f3c5654494a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:17.595177 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 390/770] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Sep 14 00:52:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:17.601124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c961525-5780-461b-956b-cf087df93492 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:17.675684 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:17.738608 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6287cb-6c91-46cf-84de-18304afb1bc5 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:17.738966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6287cb-6c91-46cf-84de-18304afb1bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:17.739162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6287cb-6c91-46cf-84de-18304afb1bc5 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:17.740641 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 391/771] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:52:17.745821 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37191825-df53-489b-8233-699a3bd775e5 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:17.746330 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37191825-df53-489b-8233-699a3bd775e5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:17.746830 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37191825-df53-489b-8233-699a3bd775e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:17.756877 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c961525-5780-461b-956b-cf087df93492 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:17.777308 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-37191825-df53-489b-8233-699a3bd775e5 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:17.778837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-37191825-df53-489b-8233-699a3bd775e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:17.801372 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c3aec-2a5d-4bb9-8350-f255d0aff48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:17.802767 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c3aec-2a5d-4bb9-8350-f255d0aff48a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:17.803519 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c3aec-2a5d-4bb9-8350-f255d0aff48a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:17.804033 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c3aec-2a5d-4bb9-8350-f255d0aff48a None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:17.805034 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c3aec-2a5d-4bb9-8350-f255d0aff48a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:17.805815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c3aec-2a5d-4bb9-8350-f255d0aff48a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:17.807426 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c3aec-2a5d-4bb9-8350-f255d0aff48a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:17.816788 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 381/772] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Sep 14 00:52:17 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:17.823043 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c3aec-2a5d-4bb9-8350-f255d0aff48a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:17.900701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 392/773] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 00:52:17.990375 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c3aec-2a5d-4bb9-8350-f255d0aff48a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:18.005725 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:18.035392 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e54cfd-d4f5-4f37-8f07-ea7bb456cd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:18.036048 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e54cfd-d4f5-4f37-8f07-ea7bb456cd8d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:18.036329 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e54cfd-d4f5-4f37-8f07-ea7bb456cd8d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:18.036608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e54cfd-d4f5-4f37-8f07-ea7bb456cd8d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:18.037315 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e54cfd-d4f5-4f37-8f07-ea7bb456cd8d None admin] RBAC: Adding query filter params (interface=public, service_id=6c26330f855d490e894e1d97ad302470, region_id=RegionOne) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:18.037624 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e54cfd-d4f5-4f37-8f07-ea7bb456cd8d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:18.039000 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e54cfd-d4f5-4f37-8f07-ea7bb456cd8d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:18.059848 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 382/774] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Sep 14 00:52:17 2022] GET /identity/v3/endpoints?service_id=6c26330f855d490e894e1d97ad302470&interface=public®ion_id=RegionOne => generated 194 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:18.067004 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37191825-df53-489b-8233-699a3bd775e5 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:18.171955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:18.208715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37191825-df53-489b-8233-699a3bd775e5 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:18.249132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c543d-b4a7-4a66-9746-c09f7533825e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:18.250235 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c543d-b4a7-4a66-9746-c09f7533825e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:18.250523 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c543d-b4a7-4a66-9746-c09f7533825e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:18.250913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c543d-b4a7-4a66-9746-c09f7533825e None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:18.251534 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23c543d-b4a7-4a66-9746-c09f7533825e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:18.252947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23c543d-b4a7-4a66-9746-c09f7533825e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:18.262914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 393/775] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Sep 14 00:52:18 2022] GET /identity/v3/services => generated 772 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:18.273154 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e54cfd-d4f5-4f37-8f07-ea7bb456cd8d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:18.447516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e54cfd-d4f5-4f37-8f07-ea7bb456cd8d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:18.454140 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcd81b-2099-4fe0-9617-4cadb7dfd30a None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:18.454456 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcd81b-2099-4fe0-9617-4cadb7dfd30a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:18.454737 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcd81b-2099-4fe0-9617-4cadb7dfd30a None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:18.456512 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 394/776] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:18.465525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d8f47-e7b3-4bda-8868-70627db253a5 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:18.466179 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d8f47-e7b3-4bda-8868-70627db253a5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:18.466738 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d8f47-e7b3-4bda-8868-70627db253a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:18.502855 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-8d7d8f47-e7b3-4bda-8868-70627db253a5 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:18.504585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8d7d8f47-e7b3-4bda-8868-70627db253a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:18.527643 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e406df9d-fa28-42a0-9000-8df60472bd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:18.529734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406df9d-fa28-42a0-9000-8df60472bd20 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:18.533687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406df9d-fa28-42a0-9000-8df60472bd20 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:18.534370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406df9d-fa28-42a0-9000-8df60472bd20 None admin] PATH_INFO: `/v3/services/network` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:18.535234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e406df9d-fa28-42a0-9000-8df60472bd20 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:18.537405 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e406df9d-fa28-42a0-9000-8df60472bd20 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:18.564646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-e406df9d-fa28-42a0-9000-8df60472bd20 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 00:52:18.564646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:18.564646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:52:18.564646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:52:18.564646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:52:18.564646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:52:18.564646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:52:18.564646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:18.564646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 00:52:18.564646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:18.564646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:18.564646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:18.564646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:18.564646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:18.564646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:18.564646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:18.564646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:18.566371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:18.566371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:18.566371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:18.566371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:18.566371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:18.566371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:18.566371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:52:18.566371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:52:18.566371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:52:18.566371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:52:18.566371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:18.566371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:18.566371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:18.566371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:18.566371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:18.566371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:18.567224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:18.567224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:18.567224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:18.567224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:18.567224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:18.567224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:18.567224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:18.567224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:18.567224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:18.567224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:18.567224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:52:18.567224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:18.567224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 00:52:18.567224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:18.576656 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 383/777] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:18 2022] GET /identity/v3/services/network => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:18.582626 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e406df9d-fa28-42a0-9000-8df60472bd20 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:18.648279 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 395/778] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 00:52:18.654871 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:18.732831 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e406df9d-fa28-42a0-9000-8df60472bd20 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:18.776277 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65568e57-ace6-42b2-a609-13f18807e5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:18.777371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65568e57-ace6-42b2-a609-13f18807e5ef None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:18.777859 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65568e57-ace6-42b2-a609-13f18807e5ef None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:18.778338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65568e57-ace6-42b2-a609-13f18807e5ef None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:18.779178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65568e57-ace6-42b2-a609-13f18807e5ef None admin] RBAC: Adding query filter params (name=network) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:18.779742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65568e57-ace6-42b2-a609-13f18807e5ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:18.781604 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65568e57-ace6-42b2-a609-13f18807e5ef None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:18.791614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 384/779] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 00:52:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:18.798302 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d8f47-e7b3-4bda-8868-70627db253a5 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:18.916500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:18.957653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d8f47-e7b3-4bda-8868-70627db253a5 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:18.999899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e92de9-ef5e-4b08-93bf-64987b931b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:19.000520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e92de9-ef5e-4b08-93bf-64987b931b57 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:19.000799 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e92de9-ef5e-4b08-93bf-64987b931b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:19.001075 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e92de9-ef5e-4b08-93bf-64987b931b57 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:19.001713 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e92de9-ef5e-4b08-93bf-64987b931b57 None admin] RBAC: Adding query filter params (type=network) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:19.002022 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e92de9-ef5e-4b08-93bf-64987b931b57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:19.003414 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e92de9-ef5e-4b08-93bf-64987b931b57 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:19.011876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 396/780] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 00:52:18 2022] GET /identity/v3/services?type=network => generated 350 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:19.017640 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65568e57-ace6-42b2-a609-13f18807e5ef None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:19.048927 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e92de9-ef5e-4b08-93bf-64987b931b57 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:19.169866 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e92de9-ef5e-4b08-93bf-64987b931b57 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:19.207436 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65568e57-ace6-42b2-a609-13f18807e5ef None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:19.216327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:19.228066 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102fa38-6f0c-44aa-948d-dec0421bca70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:19.229116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102fa38-6f0c-44aa-948d-dec0421bca70 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:19.229595 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102fa38-6f0c-44aa-948d-dec0421bca70 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:19.230089 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102fa38-6f0c-44aa-948d-dec0421bca70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:19.234714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:19.245994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c102fa38-6f0c-44aa-948d-dec0421bca70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:52:19.252124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-c102fa38-6f0c-44aa-948d-dec0421bca70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:52:19.252124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:19.252124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:19.252124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:19.252124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:19.252124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:19.252124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:19.252124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:19.252124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:19.252124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:19.252124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:19.252124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:19.252124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:52:19.252124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:52:19.252124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:52:19.252124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:52:19.253063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:52:19.253063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:52:19.253063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:52:19.253063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:52:19.253063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:19.253063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:19.253063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:19.253063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:19.253063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:19.253063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:19.253063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:19.253063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:19.253063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:19.253063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:19.253063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:19.253063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:19.253908 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:19.253908 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:19.253908 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:19.253908 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:19.253908 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:52:19.253908 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:52:19.253908 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:52:19.253908 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:52:19.253908 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:52:19.253908 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:52:19.253908 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:52:19.253908 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:19.256626 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 397/781] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:52:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:52:19.262476 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102fa38-6f0c-44aa-948d-dec0421bca70 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:19.280271 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343bd983-4ceb-432e-9e2d-ff5161ab5d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:19.281681 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bd983-4ceb-432e-9e2d-ff5161ab5d2c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:19.281776 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bd983-4ceb-432e-9e2d-ff5161ab5d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:19.282034 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bd983-4ceb-432e-9e2d-ff5161ab5d2c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:19.282758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343bd983-4ceb-432e-9e2d-ff5161ab5d2c None admin] RBAC: Adding query filter params (interface=public, service_id=f458d9d381d048a49c6642c83a162152, region_id=RegionOne) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:19.283130 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343bd983-4ceb-432e-9e2d-ff5161ab5d2c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:19.284639 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343bd983-4ceb-432e-9e2d-ff5161ab5d2c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:19.298502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 385/782] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Sep 14 00:52:19 2022] GET /identity/v3/endpoints?service_id=f458d9d381d048a49c6642c83a162152&interface=public®ion_id=RegionOne => generated 194 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:19.305821 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343bd983-4ceb-432e-9e2d-ff5161ab5d2c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:19.400831 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102fa38-6f0c-44aa-948d-dec0421bca70 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:19.440478 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c2f59-7cf3-4056-ac09-563d37ef006c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:19.441448 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c2f59-7cf3-4056-ac09-563d37ef006c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:19.441956 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c2f59-7cf3-4056-ac09-563d37ef006c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:19.442520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c2f59-7cf3-4056-ac09-563d37ef006c None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:19.443508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c2f59-7cf3-4056-ac09-563d37ef006c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:19.444028 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c2f59-7cf3-4056-ac09-563d37ef006c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:19.445856 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c2f59-7cf3-4056-ac09-563d37ef006c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:19.455472 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 398/783] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:52:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:19.461805 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c2f59-7cf3-4056-ac09-563d37ef006c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:19.537046 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343bd983-4ceb-432e-9e2d-ff5161ab5d2c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:19.542948 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:19.605020 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c2f59-7cf3-4056-ac09-563d37ef006c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:19.607015 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc818a-581c-4cf6-9f70-289fe3eb39e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:19.607985 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc818a-581c-4cf6-9f70-289fe3eb39e4 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:19.608475 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc818a-581c-4cf6-9f70-289fe3eb39e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:19.608995 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc818a-581c-4cf6-9f70-289fe3eb39e4 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:19.609887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfc818a-581c-4cf6-9f70-289fe3eb39e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:19.611533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfc818a-581c-4cf6-9f70-289fe3eb39e4 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:19.627861 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 386/784] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Sep 14 00:52:19 2022] GET /identity/v3/services => generated 772 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:19.642873 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c7c4c-40f4-4ac0-9bdd-19780fb206e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:19.643541 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c7c4c-40f4-4ac0-9bdd-19780fb206e4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:19.643843 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c7c4c-40f4-4ac0-9bdd-19780fb206e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:19.644116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c7c4c-40f4-4ac0-9bdd-19780fb206e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:19.656495 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c7c4c-40f4-4ac0-9bdd-19780fb206e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:19.658512 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c7c4c-40f4-4ac0-9bdd-19780fb206e4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:19.675031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-277c7c4c-40f4-4ac0-9bdd-19780fb206e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:19.675031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:19.675031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:19.675031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:19.675031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:19.675031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:19.675031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:19.675031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:19.675031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:19.675031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:19.675031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:19.675031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:19.675031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:19.675031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:19.675031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:19.675031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:19.676150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:19.676150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:19.676150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:19.676150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:19.676150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:19.676150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:19.676150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:19.676150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:19.676150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:19.676150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:19.676150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:19.676150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:19.676150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:19.676150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:19.676150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:19.676150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:19.676150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:19.677700 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:19.677700 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:19.677700 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:19.677700 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 399/785] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:52:19.682528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc818a-581c-4cf6-9f70-289fe3eb39e4 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:19.730597 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:19.822483 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc818a-581c-4cf6-9f70-289fe3eb39e4 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:19.860525 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c268aa-5c87-4934-b3cf-8d2714503d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:19.861060 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c268aa-5c87-4934-b3cf-8d2714503d89 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:19.861338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c268aa-5c87-4934-b3cf-8d2714503d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:19.861655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c268aa-5c87-4934-b3cf-8d2714503d89 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:19.862366 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c268aa-5c87-4934-b3cf-8d2714503d89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:19.862674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c268aa-5c87-4934-b3cf-8d2714503d89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:19.864117 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c268aa-5c87-4934-b3cf-8d2714503d89 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:19.877142 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 387/786] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:19.882680 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c7c4c-40f4-4ac0-9bdd-19780fb206e4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:20.000553 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c7c4c-40f4-4ac0-9bdd-19780fb206e4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:20.011290 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:20.045425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee8b79-bcfb-4128-b160-5f3c3c534ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:20.046078 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee8b79-bcfb-4128-b160-5f3c3c534ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:20.046361 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee8b79-bcfb-4128-b160-5f3c3c534ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:20.047233 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee8b79-bcfb-4128-b160-5f3c3c534ca1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:20.059362 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee8b79-bcfb-4128-b160-5f3c3c534ca1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:20.060956 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee8b79-bcfb-4128-b160-5f3c3c534ca1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:20.080105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-b9ee8b79-bcfb-4128-b160-5f3c3c534ca1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 00:52:20.080105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:20.080105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:20.080105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:20.080105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:20.080105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:20.080105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:20.080105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:20.080105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:20.080105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:20.080105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:20.080105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:20.080105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:52:20.080105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:52:20.080105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:52:20.080105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:52:20.081137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:20.081137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:20.081137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:52:20.081137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:20.081137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:52:20.081137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:20.081137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:20.081137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:20.081137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:20.081137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:20.081137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:20.081137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:20.081137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:20.081137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:20.081137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:20.081137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:20.081137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:20.081137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:20.082508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:20.082508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:20.082508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:52:20.082508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:52:20.082508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:52:20.082508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:52:20.082508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:52:20.082508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:52:20.082508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 00:52:20.082508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:20.083330 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2d673-8329-4991-870f-e8dbb38e3ebd None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:20.083330 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2d673-8329-4991-870f-e8dbb38e3ebd None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:20.083330 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2d673-8329-4991-870f-e8dbb38e3ebd None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:20.083530 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 400/787] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Sep 14 00:52:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:20.083633 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 388/788] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:20.088575 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee8b79-bcfb-4128-b160-5f3c3c534ca1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:20.093561 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4df440-dc02-4fce-86f7-280ef330a7e0 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:20.093925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4df440-dc02-4fce-86f7-280ef330a7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:20.094377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4df440-dc02-4fce-86f7-280ef330a7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:20.140141 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-1c4df440-dc02-4fce-86f7-280ef330a7e0 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:20.142178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1c4df440-dc02-4fce-86f7-280ef330a7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:20.203141 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee8b79-bcfb-4128-b160-5f3c3c534ca1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:20.239836 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d64e88-84b5-4a38-9001-0170342f6786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:20.240569 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d64e88-84b5-4a38-9001-0170342f6786 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:20.240856 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d64e88-84b5-4a38-9001-0170342f6786 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:20.241133 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d64e88-84b5-4a38-9001-0170342f6786 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:20.241865 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d64e88-84b5-4a38-9001-0170342f6786 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:20.242170 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d64e88-84b5-4a38-9001-0170342f6786 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:20.243636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d64e88-84b5-4a38-9001-0170342f6786 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:20.266877 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 401/789] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Sep 14 00:52:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:20.272282 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d64e88-84b5-4a38-9001-0170342f6786 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:20.307660 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 389/790] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 00:52:20.308985 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20340572-cba4-433e-bd90-9f1aa3eb9e6a None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:20.309273 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20340572-cba4-433e-bd90-9f1aa3eb9e6a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:20.309550 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20340572-cba4-433e-bd90-9f1aa3eb9e6a None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:20.311060 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 390/791] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:20.321196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b97c4b-7f33-448f-836f-9eed351305a0 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:20.321484 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b97c4b-7f33-448f-836f-9eed351305a0 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:20.321757 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b97c4b-7f33-448f-836f-9eed351305a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:20.331921 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:20.355826 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-17b97c4b-7f33-448f-836f-9eed351305a0 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:20.357348 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-17b97c4b-7f33-448f-836f-9eed351305a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:20.375418 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:20.455945 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d64e88-84b5-4a38-9001-0170342f6786 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:20.495307 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 391/792] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:20.505094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e0bb0-e80f-413b-a209-022c04958d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:20.505864 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e0bb0-e80f-413b-a209-022c04958d17 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:20.506252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e0bb0-e80f-413b-a209-022c04958d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:20.506626 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e0bb0-e80f-413b-a209-022c04958d17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:20.580400 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e0bb0-e80f-413b-a209-022c04958d17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:20.581873 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e0bb0-e80f-413b-a209-022c04958d17 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:20.611829 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:20.622554 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b97c4b-7f33-448f-836f-9eed351305a0 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:20.707004 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 402/793] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:52:20 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 00:52:20.713679 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e0bb0-e80f-413b-a209-022c04958d17 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:20.756896 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b97c4b-7f33-448f-836f-9eed351305a0 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:20.825645 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd27bdd-c8f2-421d-b981-a6beb88848f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:20.826312 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd27bdd-c8f2-421d-b981-a6beb88848f4 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:20.826591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd27bdd-c8f2-421d-b981-a6beb88848f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:20.826865 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd27bdd-c8f2-421d-b981-a6beb88848f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:20.848520 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd27bdd-c8f2-421d-b981-a6beb88848f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:52:20.857669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-8cd27bdd-c8f2-421d-b981-a6beb88848f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:52:20.857669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:20.857669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:20.857669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:20.857669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:20.857669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:20.857669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:20.857669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:20.857669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:20.857669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:20.857669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:20.857669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:20.857669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:52:20.857669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:52:20.857669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:52:20.857669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:52:20.858630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:52:20.858630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:52:20.858630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:52:20.858630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:52:20.858630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:20.858630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:20.858630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:20.858630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:20.858630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:20.858630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:20.858630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:20.858630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:20.858630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:20.858630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:20.858630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:20.858630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:20.860250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:20.860250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:20.860250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:20.860250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:20.860250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:52:20.860250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:52:20.860250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:52:20.860250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:52:20.860250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:52:20.860250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:52:20.860250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:52:20.860250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:20.866022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 392/794] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 00:52:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:20.866876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd27bdd-c8f2-421d-b981-a6beb88848f4 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:20.891767 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:20.901648 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e0bb0-e80f-413b-a209-022c04958d17 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:20.934674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:20.944710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2db4345-6435-4864-b18d-f51a264f4d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:20.945432 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db4345-6435-4864-b18d-f51a264f4d73 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:20.945788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db4345-6435-4864-b18d-f51a264f4d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:20.946139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db4345-6435-4864-b18d-f51a264f4d73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:20.958580 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2db4345-6435-4864-b18d-f51a264f4d73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:20.960404 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2db4345-6435-4864-b18d-f51a264f4d73 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:20.976983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-e2db4345-6435-4864-b18d-f51a264f4d73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:20.976983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:20.976983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:20.976983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:20.976983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:20.976983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:20.976983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:20.976983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:20.976983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:20.976983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:20.976983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:20.976983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:20.976983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:20.976983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:20.976983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:20.976983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:20.977951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:20.977951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:20.977951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:20.977951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:20.977951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:20.977951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:20.977951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:20.977951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:20.977951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:20.977951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:20.977951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:20.977951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:20.977951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:20.977951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:20.977951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:20.977951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:20.977951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:20.978868 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:20.978868 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:20.978868 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:20.980889 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 403/795] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:52:20.981919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2db4345-6435-4864-b18d-f51a264f4d73 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:20.994969 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd27bdd-c8f2-421d-b981-a6beb88848f4 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:21.035250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcb34d-ef0d-44c6-abca-21aaf87d6a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:21.036549 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcb34d-ef0d-44c6-abca-21aaf87d6a64 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:21.036905 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcb34d-ef0d-44c6-abca-21aaf87d6a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:21.037255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcb34d-ef0d-44c6-abca-21aaf87d6a64 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:21.038002 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcb34d-ef0d-44c6-abca-21aaf87d6a64 None admin] RBAC: Adding query filter params (name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:21.038378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcb34d-ef0d-44c6-abca-21aaf87d6a64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:21.040786 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcb34d-ef0d-44c6-abca-21aaf87d6a64 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:21.057226 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 393/796] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 00:52:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:21.057226 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcb34d-ef0d-44c6-abca-21aaf87d6a64 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:21.120134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2db4345-6435-4864-b18d-f51a264f4d73 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:21.159649 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:21.166189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245dc7a-c92e-42f6-990d-0ef9ce157aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:21.166835 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245dc7a-c92e-42f6-990d-0ef9ce157aae None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:21.167115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245dc7a-c92e-42f6-990d-0ef9ce157aae None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:21.167388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245dc7a-c92e-42f6-990d-0ef9ce157aae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:21.182113 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3245dc7a-c92e-42f6-990d-0ef9ce157aae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:52:21.188116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-3245dc7a-c92e-42f6-990d-0ef9ce157aae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:52:21.188116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:21.188116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:21.188116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:21.188116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:21.188116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:21.188116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:21.188116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:21.188116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:21.188116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:21.188116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:21.188116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:21.188116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:52:21.188116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:52:21.188116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:52:21.188116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:52:21.189181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:52:21.189181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:52:21.189181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:52:21.189181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:52:21.189181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:21.189181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:21.189181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:21.189181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:21.189181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:21.189181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:21.189181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:21.189181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:21.189181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:21.189181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:21.189181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:21.189181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:21.190041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:21.190041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:21.190041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:21.190041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:21.190041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:52:21.190041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:52:21.190041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:52:21.190041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:52:21.190041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:52:21.190041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:52:21.190041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:52:21.190041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:21.190041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 404/797] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:52:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:21.190736 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245dc7a-c92e-42f6-990d-0ef9ce157aae None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:21.197298 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcb34d-ef0d-44c6-abca-21aaf87d6a64 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:21.235538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f01bf-464d-4f8e-9b21-7a81ac68ccb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:21.236107 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f01bf-464d-4f8e-9b21-7a81ac68ccb7 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:21.236385 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f01bf-464d-4f8e-9b21-7a81ac68ccb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:21.236654 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f01bf-464d-4f8e-9b21-7a81ac68ccb7 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:21.237165 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:21.237703 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f01bf-464d-4f8e-9b21-7a81ac68ccb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:21.237941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f01bf-464d-4f8e-9b21-7a81ac68ccb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:21.239825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f01bf-464d-4f8e-9b21-7a81ac68ccb7 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:21.257352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 394/798] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:21.257352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb79cd-39b0-4952-a43c-403dd2fabd83 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:21.257352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb79cd-39b0-4952-a43c-403dd2fabd83 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:21.257352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb79cd-39b0-4952-a43c-403dd2fabd83 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:21.257352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 395/799] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:21.257352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb79cd-39b0-4952-a43c-403dd2fabd83 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:21.309000 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245dc7a-c92e-42f6-990d-0ef9ce157aae None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:21.344812 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e86db-dccb-4da8-ac44-d6a0e6870f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:21.345446 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e86db-dccb-4da8-ac44-d6a0e6870f99 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:21.345722 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e86db-dccb-4da8-ac44-d6a0e6870f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:21.345997 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e86db-dccb-4da8-ac44-d6a0e6870f99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:21.356617 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e86db-dccb-4da8-ac44-d6a0e6870f99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:21.358062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e86db-dccb-4da8-ac44-d6a0e6870f99 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:21.368001 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb79cd-39b0-4952-a43c-403dd2fabd83 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:21.372899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-f59e86db-dccb-4da8-ac44-d6a0e6870f99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:21.372899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:21.372899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:21.372899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:21.372899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:21.372899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:21.372899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:21.372899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:21.372899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:21.372899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:21.372899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:21.372899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:21.372899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:21.372899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:21.372899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:21.372899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:21.373917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:21.373917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:21.373917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:21.373917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:21.373917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:21.373917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:21.373917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:21.373917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:21.373917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:21.373917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:21.373917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:21.373917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:21.373917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:21.373917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:21.373917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:21.373917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:21.373917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:21.375400 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:21.375400 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:21.375400 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:21.376781 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 405/800] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:21.377699 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e86db-dccb-4da8-ac44-d6a0e6870f99 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:21.407474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a83f3ed-c212-4427-ac9b-b702f0c4f693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:21.408110 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83f3ed-c212-4427-ac9b-b702f0c4f693 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:21.408390 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83f3ed-c212-4427-ac9b-b702f0c4f693 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:21.408659 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83f3ed-c212-4427-ac9b-b702f0c4f693 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:21.409302 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a83f3ed-c212-4427-ac9b-b702f0c4f693 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:21.409605 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a83f3ed-c212-4427-ac9b-b702f0c4f693 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:21.410969 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a83f3ed-c212-4427-ac9b-b702f0c4f693 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:21.412364 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:21.420585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 396/801] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:52:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:21.421977 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3bb4a-c400-4a15-ad72-35936a068c2b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:21.422319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3bb4a-c400-4a15-ad72-35936a068c2b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:21.422717 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3bb4a-c400-4a15-ad72-35936a068c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:21.443652 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e7f3bb4a-c400-4a15-ad72-35936a068c2b None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:21.444842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e7f3bb4a-c400-4a15-ad72-35936a068c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:21.488361 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:21.498399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e86db-dccb-4da8-ac44-d6a0e6870f99 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:21.517494 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 397/802] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:21.518229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f3bb4a-c400-4a15-ad72-35936a068c2b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:21.545378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf03a8b5-eb6a-4fb6-be81-5699c09b761e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:21.546679 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf03a8b5-eb6a-4fb6-be81-5699c09b761e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:21.547509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf03a8b5-eb6a-4fb6-be81-5699c09b761e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:21.548141 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf03a8b5-eb6a-4fb6-be81-5699c09b761e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:21.557165 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf03a8b5-eb6a-4fb6-be81-5699c09b761e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:21.559007 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf03a8b5-eb6a-4fb6-be81-5699c09b761e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:21.574136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-bf03a8b5-eb6a-4fb6-be81-5699c09b761e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:21.574136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:21.574136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:21.574136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:21.574136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:21.574136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:21.574136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:21.574136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:21.574136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:21.574136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:21.574136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:21.574136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:21.574136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:52:21.574136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:52:21.574136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:52:21.575035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:52:21.575035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:21.575035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:21.575035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:21.575035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:21.575035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:21.575035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:21.575035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:21.575035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:21.575035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:21.575035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:21.575035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:21.575035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:21.575035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:21.575035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:21.575035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:21.575035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:21.575967 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:52:21.575967 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:52:21.575967 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:52:21.575967 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:52:21.575967 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:52:21.575967 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:52:21.575967 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:21.575967 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:21.583131 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 406/803] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:21.583131 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf03a8b5-eb6a-4fb6-be81-5699c09b761e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:21.641050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f3bb4a-c400-4a15-ad72-35936a068c2b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:21.676772 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:21.689169 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72977cf-4e04-457d-8b26-08131c880fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:21.690630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72977cf-4e04-457d-8b26-08131c880fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:21.691067 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72977cf-4e04-457d-8b26-08131c880fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:21.691797 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72977cf-4e04-457d-8b26-08131c880fb7 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:21.692597 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72977cf-4e04-457d-8b26-08131c880fb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:21.693296 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72977cf-4e04-457d-8b26-08131c880fb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:21.695514 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72977cf-4e04-457d-8b26-08131c880fb7 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:21.696914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf03a8b5-eb6a-4fb6-be81-5699c09b761e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:21.715051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 398/804] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:21.715051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72977cf-4e04-457d-8b26-08131c880fb7 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:21.748723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b2b48-bfc6-497d-829a-195beab7b548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:21.750120 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b2b48-bfc6-497d-829a-195beab7b548 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:21.750789 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b2b48-bfc6-497d-829a-195beab7b548 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:21.751274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b2b48-bfc6-497d-829a-195beab7b548 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:21.767052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b2b48-bfc6-497d-829a-195beab7b548 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:21.769351 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b2b48-bfc6-497d-829a-195beab7b548 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:21.789612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-628b2b48-bfc6-497d-829a-195beab7b548 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:21.789612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:21.789612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:21.789612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:21.789612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:21.789612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:21.789612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:21.789612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:21.789612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:21.789612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:21.789612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:21.789612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:21.789612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:21.789612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:21.789612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:21.789612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:21.790833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:21.790833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:21.790833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:21.790833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:21.790833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:21.790833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:21.790833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:21.790833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:21.790833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:21.790833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:21.790833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:21.790833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:21.790833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:21.790833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:21.790833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:21.790833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:21.790833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:21.791997 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:21.791997 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:21.791997 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:21.798644 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 407/805] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:21.798644 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b2b48-bfc6-497d-829a-195beab7b548 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:21.800179 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:21.831411 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72977cf-4e04-457d-8b26-08131c880fb7 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:21.865752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538725c1-50ae-4f8c-83e6-8ac3b498eb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:21.866362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538725c1-50ae-4f8c-83e6-8ac3b498eb14 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:21.866644 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538725c1-50ae-4f8c-83e6-8ac3b498eb14 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:21.866916 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538725c1-50ae-4f8c-83e6-8ac3b498eb14 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:21.867638 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538725c1-50ae-4f8c-83e6-8ac3b498eb14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:21.867944 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538725c1-50ae-4f8c-83e6-8ac3b498eb14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:21.869390 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538725c1-50ae-4f8c-83e6-8ac3b498eb14 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:21.887460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 399/806] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:52:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:21.887460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538725c1-50ae-4f8c-83e6-8ac3b498eb14 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:21.915047 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b2b48-bfc6-497d-829a-195beab7b548 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:21.935725 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:21.953339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4acd91-ea7e-4b7c-99a1-dff066730192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:21.954215 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4acd91-ea7e-4b7c-99a1-dff066730192 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:21.954498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4acd91-ea7e-4b7c-99a1-dff066730192 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:21.954767 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4acd91-ea7e-4b7c-99a1-dff066730192 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:21.962697 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4acd91-ea7e-4b7c-99a1-dff066730192 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:21.964165 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4acd91-ea7e-4b7c-99a1-dff066730192 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:21.976426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-db4acd91-ea7e-4b7c-99a1-dff066730192 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 00:52:21.976426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:21.976426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:21.976426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:21.976426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:21.976426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:21.976426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:21.976426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:21.976426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:21.976426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:21.976426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:21.976426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:21.976426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:52:21.976426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:52:21.976426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:52:21.976426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:52:21.977407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:21.977407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:21.977407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:52:21.977407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:21.977407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:52:21.977407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:21.977407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:21.977407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:21.977407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:21.977407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:21.977407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:21.977407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:21.977407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:21.977407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:21.977407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:21.977407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:21.977407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:21.977407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:21.978354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:21.978354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:21.978354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:52:21.978354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:52:21.978354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:52:21.978354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:52:21.978354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:52:21.978354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:52:21.978354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 00:52:21.978354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:21.978354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 408/807] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Sep 14 00:52:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:21.978354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4acd91-ea7e-4b7c-99a1-dff066730192 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:21.998109 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538725c1-50ae-4f8c-83e6-8ac3b498eb14 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:22.038168 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50b629-8f1c-423e-ba5e-e9f4e3e737ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:22.038738 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50b629-8f1c-423e-ba5e-e9f4e3e737ba None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:22.039014 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50b629-8f1c-423e-ba5e-e9f4e3e737ba None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:22.039287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50b629-8f1c-423e-ba5e-e9f4e3e737ba None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:22.039919 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50b629-8f1c-423e-ba5e-e9f4e3e737ba None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:22.041521 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50b629-8f1c-423e-ba5e-e9f4e3e737ba None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:22.051909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-bd50b629-8f1c-423e-ba5e-e9f4e3e737ba None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 00:52:22.051909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:22.051909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:52:22.051909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:52:22.051909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:52:22.051909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:52:22.051909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:52:22.051909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:22.051909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 00:52:22.051909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:22.051909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:22.051909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:22.051909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:22.051909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:22.051909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:22.051909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:22.051909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:22.053181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:22.053181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:22.053181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:22.053181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:22.053181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:22.053181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:22.053181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:52:22.053181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:52:22.053181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:52:22.053181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:52:22.053181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:22.053181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:22.053181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:22.053181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:22.053181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:22.053181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:22.054219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:22.054219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:22.054219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:22.054219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:22.054219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:22.054219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:22.054219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:22.054219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:22.054219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:22.054219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:22.054219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:52:22.054219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:22.054219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 00:52:22.054219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:22.054219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 400/808] 10.222.0.42 () {56 vars in 1129 bytes} [Wed Sep 14 00:52:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:22.055049 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50b629-8f1c-423e-ba5e-e9f4e3e737ba None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:22.062967 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:22.107870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4acd91-ea7e-4b7c-99a1-dff066730192 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:22.146579 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda0683-b68e-4af2-9cae-1111fd00fc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:22.147184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda0683-b68e-4af2-9cae-1111fd00fc02 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:22.147465 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda0683-b68e-4af2-9cae-1111fd00fc02 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:22.147764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda0683-b68e-4af2-9cae-1111fd00fc02 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:22.148404 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfda0683-b68e-4af2-9cae-1111fd00fc02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:22.148704 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfda0683-b68e-4af2-9cae-1111fd00fc02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:22.150076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfda0683-b68e-4af2-9cae-1111fd00fc02 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:22.161287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50b629-8f1c-423e-ba5e-e9f4e3e737ba None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:22.162103 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 409/809] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:22.162929 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda0683-b68e-4af2-9cae-1111fd00fc02 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:22.176351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:22.200177 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2303747-a716-4ed6-a720-13b909b7613b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:22.200781 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2303747-a716-4ed6-a720-13b909b7613b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:22.201064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2303747-a716-4ed6-a720-13b909b7613b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:22.201340 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2303747-a716-4ed6-a720-13b909b7613b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:22.202094 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2303747-a716-4ed6-a720-13b909b7613b None admin] RBAC: Adding query filter params (role.id=ada268ea63354a0297070369dfe3c6cc, scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3, user.id=5e0fe2e84f6748c992c9beedf2f5d49b) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:22.202430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2303747-a716-4ed6-a720-13b909b7613b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:22.204531 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2303747-a716-4ed6-a720-13b909b7613b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:22.210355 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 401/810] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:52:22 2022] GET /identity/v3/role_assignments?user.id=5e0fe2e84f6748c992c9beedf2f5d49b&scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3&role.id=ada268ea63354a0297070369dfe3c6cc => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:22.211309 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2303747-a716-4ed6-a720-13b909b7613b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:22.272635 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda0683-b68e-4af2-9cae-1111fd00fc02 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:22.301345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:22.316806 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc734d93-ddf4-4dea-a52c-e26c470da284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:22.317435 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc734d93-ddf4-4dea-a52c-e26c470da284 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:22.317725 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc734d93-ddf4-4dea-a52c-e26c470da284 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:22.317997 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc734d93-ddf4-4dea-a52c-e26c470da284 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:22.318749 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc734d93-ddf4-4dea-a52c-e26c470da284 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:22.319058 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc734d93-ddf4-4dea-a52c-e26c470da284 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:22.320562 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc734d93-ddf4-4dea-a52c-e26c470da284 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:22.338682 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2303747-a716-4ed6-a720-13b909b7613b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:22.346057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 410/811] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Sep 14 00:52:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:22.346057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93e99d-fa0e-4cbd-a138-d9be087fa130 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:22.346057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93e99d-fa0e-4cbd-a138-d9be087fa130 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:22.346057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93e99d-fa0e-4cbd-a138-d9be087fa130 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:22.346057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 411/812] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:22.346057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93e99d-fa0e-4cbd-a138-d9be087fa130 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:22.385635 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee7217-875f-43d7-bf59-4b0e88c7db2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:22.386904 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee7217-875f-43d7-bf59-4b0e88c7db2e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:22.387383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee7217-875f-43d7-bf59-4b0e88c7db2e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:22.387868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee7217-875f-43d7-bf59-4b0e88c7db2e None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:22.388783 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee7217-875f-43d7-bf59-4b0e88c7db2e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:22.389090 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee7217-875f-43d7-bf59-4b0e88c7db2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:22.391997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee7217-875f-43d7-bf59-4b0e88c7db2e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:22.397236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 402/813] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Sep 14 00:52:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:22.398137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee7217-875f-43d7-bf59-4b0e88c7db2e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:22.445935 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:22.461861 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93e99d-fa0e-4cbd-a138-d9be087fa130 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:22.499732 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc68705-eb91-4da0-b018-b8ede491d4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:22.500369 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc68705-eb91-4da0-b018-b8ede491d4c3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:22.500653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc68705-eb91-4da0-b018-b8ede491d4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:22.500950 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc68705-eb91-4da0-b018-b8ede491d4c3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:22.508283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc68705-eb91-4da0-b018-b8ede491d4c3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:22.509726 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc68705-eb91-4da0-b018-b8ede491d4c3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:22.518728 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee7217-875f-43d7-bf59-4b0e88c7db2e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:22.522046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-7dc68705-eb91-4da0-b018-b8ede491d4c3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 00:52:22.522046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:22.522046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:22.522046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:22.522046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:22.522046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:22.522046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:22.522046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:22.522046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:22.522046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:22.522046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:22.522046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:22.522046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:52:22.522046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:52:22.522046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:52:22.522046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:52:22.522974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:22.522974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:22.522974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:52:22.522974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:22.522974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:52:22.522974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:22.522974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:22.522974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:22.522974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:22.522974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:22.522974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:22.522974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:22.522974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:22.522974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:22.522974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:22.522974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:22.522974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:22.522974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:22.528762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:22.528762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:22.528762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:52:22.528762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:52:22.528762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:52:22.528762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:52:22.528762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:52:22.528762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:52:22.528762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 00:52:22.528762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:22.528762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 412/814] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:52:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:22.528762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4597f-225d-403b-9057-8befc7979844 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:22.528762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4597f-225d-403b-9057-8befc7979844 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:22.529477 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4597f-225d-403b-9057-8befc7979844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:22.544568 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-14d4597f-225d-403b-9057-8befc7979844 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:22.545755 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-14d4597f-225d-403b-9057-8befc7979844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:22.560057 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfdbdd-3bce-449c-8e19-a40ee875c2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:22.560746 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfdbdd-3bce-449c-8e19-a40ee875c2fa None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:22.561031 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfdbdd-3bce-449c-8e19-a40ee875c2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:22.561308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfdbdd-3bce-449c-8e19-a40ee875c2fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:22.573464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfdbdd-3bce-449c-8e19-a40ee875c2fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:22.574919 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfdbdd-3bce-449c-8e19-a40ee875c2fa None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:22.584877 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:22.590674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-e8cfdbdd-3bce-449c-8e19-a40ee875c2fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:22.590674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:22.590674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:22.590674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:22.590674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:22.590674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:22.590674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:22.590674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:22.590674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:22.590674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:22.590674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:22.590674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:22.590674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:22.590674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:22.590674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:22.590674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:22.591841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:22.591841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:22.591841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:22.591841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:22.591841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:22.591841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:22.591841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:22.591841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:22.591841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:22.591841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:22.591841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:22.591841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:22.591841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:22.591841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:22.591841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:22.591841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:22.591841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:22.592752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:22.592752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:22.592752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:22.592752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 403/815] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:22.593384 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfdbdd-3bce-449c-8e19-a40ee875c2fa None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:22.625737 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 413/816] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:22.626936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4597f-225d-403b-9057-8befc7979844 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:22.699338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:22.710727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfdbdd-3bce-449c-8e19-a40ee875c2fa None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:22.753593 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3f90d-4d9c-4853-8a6b-31f159faffc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:22.754222 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3f90d-4d9c-4853-8a6b-31f159faffc9 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:22.754506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3f90d-4d9c-4853-8a6b-31f159faffc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:22.754782 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3f90d-4d9c-4853-8a6b-31f159faffc9 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:22.755693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3f90d-4d9c-4853-8a6b-31f159faffc9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:22.756010 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3f90d-4d9c-4853-8a6b-31f159faffc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:22.758076 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3f90d-4d9c-4853-8a6b-31f159faffc9 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:22.759951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4597f-225d-403b-9057-8befc7979844 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:22.764731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 404/817] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Sep 14 00:52:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:22.765940 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3f90d-4d9c-4853-8a6b-31f159faffc9 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:22.805291 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6c710-1816-4311-809c-3e903fffa29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:22.805942 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6c710-1816-4311-809c-3e903fffa29e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:22.806226 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6c710-1816-4311-809c-3e903fffa29e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:22.806505 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6c710-1816-4311-809c-3e903fffa29e None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:22.807259 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d6c710-1816-4311-809c-3e903fffa29e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:22.807650 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d6c710-1816-4311-809c-3e903fffa29e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:22.809278 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d6c710-1816-4311-809c-3e903fffa29e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:22.837891 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 414/818] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Sep 14 00:52:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:22.837891 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6c710-1816-4311-809c-3e903fffa29e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:22.873083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:22.883511 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3f90d-4d9c-4853-8a6b-31f159faffc9 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:22.922862 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d671ab7-e2ea-47b5-8e83-822696187933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:22.923499 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d671ab7-e2ea-47b5-8e83-822696187933 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:22.923812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d671ab7-e2ea-47b5-8e83-822696187933 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:22.924086 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d671ab7-e2ea-47b5-8e83-822696187933 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:22.924768 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d671ab7-e2ea-47b5-8e83-822696187933 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:22.925071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d671ab7-e2ea-47b5-8e83-822696187933 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:22.926497 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d671ab7-e2ea-47b5-8e83-822696187933 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:22.940921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 405/819] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:22.943715 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d671ab7-e2ea-47b5-8e83-822696187933 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:22.958802 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:23.061069 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d671ab7-e2ea-47b5-8e83-822696187933 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:23.075781 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6c710-1816-4311-809c-3e903fffa29e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:23.099424 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a97c1e-1431-408c-b71b-76574f709a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:23.100031 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a97c1e-1431-408c-b71b-76574f709a88 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:23.100313 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a97c1e-1431-408c-b71b-76574f709a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:23.100589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a97c1e-1431-408c-b71b-76574f709a88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:23.111505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a97c1e-1431-408c-b71b-76574f709a88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:23.112973 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a97c1e-1431-408c-b71b-76574f709a88 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:23.115205 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253bb9a1-3e15-493d-8179-03ce459059cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:23.116144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bb9a1-3e15-493d-8179-03ce459059cb None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:23.116431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bb9a1-3e15-493d-8179-03ce459059cb None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:23.116705 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bb9a1-3e15-493d-8179-03ce459059cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:23.117403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253bb9a1-3e15-493d-8179-03ce459059cb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:23.118755 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253bb9a1-3e15-493d-8179-03ce459059cb None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:23.127383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-04a97c1e-1431-408c-b71b-76574f709a88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:23.127383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:23.127383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:23.127383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:23.127383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:23.127383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:23.127383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:23.127383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:23.127383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:23.127383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:23.127383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:23.127383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:23.127383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:23.127383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:23.127383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:23.127383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:23.128370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:23.128370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:23.128370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:23.128370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:23.128370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:23.128370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:23.128370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:23.128370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:23.128370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:23.128370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:23.128370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:23.128370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:23.128370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:23.128370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:23.128370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:23.128370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:23.128370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:23.129267 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:23.129267 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:23.129267 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:23.133217 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 406/820] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:23.133827 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a97c1e-1431-408c-b71b-76574f709a88 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:23.149420 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 415/821] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 00:52:22 2022] POST /identity/v3/endpoints => generated 346 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:52:23.150369 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253bb9a1-3e15-493d-8179-03ce459059cb None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:23.219840 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:23.262200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a97c1e-1431-408c-b71b-76574f709a88 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:23.273648 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:23.303958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253bb9a1-3e15-493d-8179-03ce459059cb None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:23.311075 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772fe81e-05e7-4b63-b08c-8d1d417b9fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:23.311820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fe81e-05e7-4b63-b08c-8d1d417b9fff None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:23.312217 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fe81e-05e7-4b63-b08c-8d1d417b9fff None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:23.312505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fe81e-05e7-4b63-b08c-8d1d417b9fff None admin] PATH_INFO: `/v3/services/network` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:23.313278 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772fe81e-05e7-4b63-b08c-8d1d417b9fff None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:23.314708 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772fe81e-05e7-4b63-b08c-8d1d417b9fff None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:23.324828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-772fe81e-05e7-4b63-b08c-8d1d417b9fff None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 00:52:23.324828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:23.324828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:52:23.324828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:52:23.324828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:52:23.324828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:52:23.324828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:52:23.324828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:23.324828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 00:52:23.324828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:23.324828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:23.324828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:23.324828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:23.324828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:23.324828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:23.324828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:23.324828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:23.325925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:23.325925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:23.325925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:23.325925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:23.325925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:23.325925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:23.325925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:52:23.325925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:52:23.325925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:52:23.325925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:52:23.325925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:23.325925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:23.325925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:23.325925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:23.325925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:23.325925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:23.326775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:23.326775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:23.326775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:23.326775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:23.326775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:23.326775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:23.326775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:23.326775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:23.326775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:23.326775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:23.326775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:52:23.326775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:23.326775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 00:52:23.326775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:23.326775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 407/822] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:23 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:23.328843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772fe81e-05e7-4b63-b08c-8d1d417b9fff None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:23.356077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c71068-f1ce-4d4d-8557-f73220e5f850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:23.357615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c71068-f1ce-4d4d-8557-f73220e5f850 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:23.358494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c71068-f1ce-4d4d-8557-f73220e5f850 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:23.359031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c71068-f1ce-4d4d-8557-f73220e5f850 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:23.370760 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c71068-f1ce-4d4d-8557-f73220e5f850 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:23.373050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c71068-f1ce-4d4d-8557-f73220e5f850 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:23.391334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-53c71068-f1ce-4d4d-8557-f73220e5f850 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:23.391334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:23.391334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:23.391334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:23.391334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:23.391334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:23.391334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:23.391334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:23.391334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:23.391334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:23.391334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:23.391334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:23.391334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:52:23.391334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:52:23.391334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:52:23.392403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:52:23.392403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:23.392403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:23.392403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:23.392403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:23.392403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:23.392403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:23.392403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:23.392403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:23.392403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:23.392403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:23.392403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:23.392403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:23.392403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:23.392403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:23.392403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:23.392403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:23.393316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:52:23.393316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:52:23.393316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:52:23.393316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:52:23.393316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:52:23.393316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:52:23.393316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:23.393316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:23.393316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 416/823] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:23.393809 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c71068-f1ce-4d4d-8557-f73220e5f850 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:23.447368 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772fe81e-05e7-4b63-b08c-8d1d417b9fff None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:23.490359 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:23.498066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417622c-4a6c-462c-b7f4-b5ce514f2596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:23.498987 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417622c-4a6c-462c-b7f4-b5ce514f2596 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:23.500054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417622c-4a6c-462c-b7f4-b5ce514f2596 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:23.500730 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417622c-4a6c-462c-b7f4-b5ce514f2596 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:23.501578 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4417622c-4a6c-462c-b7f4-b5ce514f2596 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:23.502099 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4417622c-4a6c-462c-b7f4-b5ce514f2596 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:23.503718 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4417622c-4a6c-462c-b7f4-b5ce514f2596 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:23.516897 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 408/824] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:23.521891 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c71068-f1ce-4d4d-8557-f73220e5f850 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:23.522636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417622c-4a6c-462c-b7f4-b5ce514f2596 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:23.557318 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885e05c-1e9a-4b62-9d4a-2548feca9e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:23.557934 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885e05c-1e9a-4b62-9d4a-2548feca9e37 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:23.558221 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885e05c-1e9a-4b62-9d4a-2548feca9e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:23.558585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885e05c-1e9a-4b62-9d4a-2548feca9e37 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:23.559232 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885e05c-1e9a-4b62-9d4a-2548feca9e37 None admin] RBAC: Adding query filter params (name=network) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:23.559538 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885e05c-1e9a-4b62-9d4a-2548feca9e37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:23.560937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e885e05c-1e9a-4b62-9d4a-2548feca9e37 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:23.566558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 417/825] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 00:52:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:23.567390 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885e05c-1e9a-4b62-9d4a-2548feca9e37 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:23.576229 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:23.645336 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417622c-4a6c-462c-b7f4-b5ce514f2596 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:23.673364 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885e05c-1e9a-4b62-9d4a-2548feca9e37 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:23.682737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589491a2-d0dc-4d60-9a2f-efadb42a800c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:23.683689 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589491a2-d0dc-4d60-9a2f-efadb42a800c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:23.684119 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589491a2-d0dc-4d60-9a2f-efadb42a800c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:23.684540 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589491a2-d0dc-4d60-9a2f-efadb42a800c None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:23.685381 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589491a2-d0dc-4d60-9a2f-efadb42a800c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:23.685833 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589491a2-d0dc-4d60-9a2f-efadb42a800c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:23.688561 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589491a2-d0dc-4d60-9a2f-efadb42a800c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:23.706693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 409/826] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:52:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:23.707897 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589491a2-d0dc-4d60-9a2f-efadb42a800c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:23.708545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a50a43c-a96d-4169-ac04-11d7a500817b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:23.709185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50a43c-a96d-4169-ac04-11d7a500817b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:23.709478 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50a43c-a96d-4169-ac04-11d7a500817b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:23.709757 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50a43c-a96d-4169-ac04-11d7a500817b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:23.716761 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a50a43c-a96d-4169-ac04-11d7a500817b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:23.718806 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a50a43c-a96d-4169-ac04-11d7a500817b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:23.731937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-8a50a43c-a96d-4169-ac04-11d7a500817b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:23.731937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:23.731937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:23.731937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:23.731937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:23.731937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:23.731937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:23.731937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:23.731937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:23.731937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:23.731937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:23.731937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:23.731937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:52:23.731937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:52:23.731937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:52:23.733586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:52:23.733586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:23.733586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:23.733586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:23.733586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:23.733586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:23.733586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:23.733586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:23.733586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:23.733586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:23.733586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:23.733586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:23.733586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:23.733586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:23.733586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:23.733586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:23.733586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:23.735888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:52:23.735888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:52:23.735888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:52:23.735888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:52:23.735888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:52:23.735888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:52:23.735888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:23.735888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:23.735888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 418/827] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:23.735888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a50a43c-a96d-4169-ac04-11d7a500817b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:23.775191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:23.802519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:23.834391 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589491a2-d0dc-4d60-9a2f-efadb42a800c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:23.838507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a50a43c-a96d-4169-ac04-11d7a500817b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:23.872464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e99bbf-fc33-4477-9574-ea454f63b342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:23.873285 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc94e7-17a3-4537-a47a-aa7497afe232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:23.873700 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e99bbf-fc33-4477-9574-ea454f63b342 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:23.873948 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc94e7-17a3-4537-a47a-aa7497afe232 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:23.874234 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc94e7-17a3-4537-a47a-aa7497afe232 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:23.874586 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e99bbf-fc33-4477-9574-ea454f63b342 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:23.874704 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc94e7-17a3-4537-a47a-aa7497afe232 None admin] PATH_INFO: `/v3/projects/fda69e8c2d5e466dbc1244b2f76296a3/users/ceccc909529042539dd2d5015058d8f7/roles/7a33c2ceb92a478f81753573f8ece1bd` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:23.875162 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e99bbf-fc33-4477-9574-ea454f63b342 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:23.875989 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e99bbf-fc33-4477-9574-ea454f63b342 None admin] RBAC: Adding query filter params (type=network) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:23.876441 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e99bbf-fc33-4477-9574-ea454f63b342 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:23.878161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e99bbf-fc33-4477-9574-ea454f63b342 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:23.888486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 410/828] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 00:52:23 2022] GET /identity/v3/services?type=network => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:23.889690 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e99bbf-fc33-4477-9574-ea454f63b342 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:23.891757 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cc94e7-17a3-4537-a47a-aa7497afe232 None admin] RBAC: Authorizing `identity:create_grant(project_id=fda69e8c2d5e466dbc1244b2f76296a3, user_id=ceccc909529042539dd2d5015058d8f7, role_id=7a33c2ceb92a478f81753573f8ece1bd)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:23.893132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cc94e7-17a3-4537-a47a-aa7497afe232 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:23.915374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 419/829] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 00:52:23 2022] PUT /identity/v3/projects/fda69e8c2d5e466dbc1244b2f76296a3/users/ceccc909529042539dd2d5015058d8f7/roles/7a33c2ceb92a478f81753573f8ece1bd => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:52:23.915374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc94e7-17a3-4537-a47a-aa7497afe232 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:24.029441 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc94e7-17a3-4537-a47a-aa7497afe232 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:24.057544 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:24.076819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad50f5-ccb6-47d9-b9f3-e0c5b8e727ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:24.077460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad50f5-ccb6-47d9-b9f3-e0c5b8e727ef None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:24.078271 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad50f5-ccb6-47d9-b9f3-e0c5b8e727ef None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:24.078555 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad50f5-ccb6-47d9-b9f3-e0c5b8e727ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:24.079259 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ad50f5-ccb6-47d9-b9f3-e0c5b8e727ef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:24.080725 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ad50f5-ccb6-47d9-b9f3-e0c5b8e727ef None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:24.096864 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e99bbf-fc33-4477-9574-ea454f63b342 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:24.105860 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 420/830] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 00:52:23 2022] POST /identity/v3/endpoints => generated 337 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:52:24.129761 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:24.176970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452d868-cae3-4175-a8d5-068e3d22d664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:24.177519 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452d868-cae3-4175-a8d5-068e3d22d664 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:24.177797 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452d868-cae3-4175-a8d5-068e3d22d664 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:24.178071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452d868-cae3-4175-a8d5-068e3d22d664 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:24.178823 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452d868-cae3-4175-a8d5-068e3d22d664 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:24.179133 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452d868-cae3-4175-a8d5-068e3d22d664 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:24.180797 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452d868-cae3-4175-a8d5-068e3d22d664 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:24.201820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 411/831] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:52:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:24.208385 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad50f5-ccb6-47d9-b9f3-e0c5b8e727ef None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:24.323284 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad50f5-ccb6-47d9-b9f3-e0c5b8e727ef None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:24.358168 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11b5ea-e5d3-4b33-b953-ece52205b4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:24.359351 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11b5ea-e5d3-4b33-b953-ece52205b4b0 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:24.359776 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11b5ea-e5d3-4b33-b953-ece52205b4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:24.360051 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11b5ea-e5d3-4b33-b953-ece52205b4b0 None admin] PATH_INFO: `/v3/projects/fda69e8c2d5e466dbc1244b2f76296a3/users/e52776d9b4894af9b85a24136a8255e5/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:24.361973 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f11b5ea-e5d3-4b33-b953-ece52205b4b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=fda69e8c2d5e466dbc1244b2f76296a3, user_id=e52776d9b4894af9b85a24136a8255e5, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:24.363408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f11b5ea-e5d3-4b33-b953-ece52205b4b0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:24.378603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 421/832] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 00:52:24 2022] PUT /identity/v3/projects/fda69e8c2d5e466dbc1244b2f76296a3/users/e52776d9b4894af9b85a24136a8255e5/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:52:24.656577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9ea70-5b54-44c4-a5b9-e8053a2d565b None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:24.657131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9ea70-5b54-44c4-a5b9-e8053a2d565b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:24.657432 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9ea70-5b54-44c4-a5b9-e8053a2d565b None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:24.659167 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 412/833] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:24.664729 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e1602-b047-44d9-8c1f-646ec2a0c02d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:24.665124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e1602-b047-44d9-8c1f-646ec2a0c02d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:24.665500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e1602-b047-44d9-8c1f-646ec2a0c02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:24.684369 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-3e8e1602-b047-44d9-8c1f-646ec2a0c02d None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:24.685554 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3e8e1602-b047-44d9-8c1f-646ec2a0c02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:24.712393 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:24.759756 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 422/834] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:25.034237 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9ea70-5b54-44c4-a5b9-e8053a2d565b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:25.146519 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9ea70-5b54-44c4-a5b9-e8053a2d565b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:25.182849 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b078c2-31a8-49fa-8fe1-e125975c5def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:25.183488 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b078c2-31a8-49fa-8fe1-e125975c5def None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:25.183792 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b078c2-31a8-49fa-8fe1-e125975c5def None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:25.184067 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b078c2-31a8-49fa-8fe1-e125975c5def None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:25.184650 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b078c2-31a8-49fa-8fe1-e125975c5def None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:25.186023 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b078c2-31a8-49fa-8fe1-e125975c5def None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:25.196230 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-74b078c2-31a8-49fa-8fe1-e125975c5def None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 00:52:25.196230 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:25.196230 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:52:25.196230 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:52:25.196230 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:52:25.196230 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:52:25.196230 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:52:25.196230 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:25.196230 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 00:52:25.196230 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:25.196230 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:25.196230 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:25.196230 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:25.196230 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:25.196230 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:25.196230 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:25.197229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:25.197229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:25.197229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:25.197229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:25.197229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:25.197229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:25.197229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:25.197229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:52:25.197229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:52:25.197229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:52:25.197229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:52:25.197229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:25.197229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:25.197229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:25.197229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:25.197229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:25.197229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:25.198137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:25.198137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:25.198137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:25.198137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:25.198137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:25.198137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:25.198137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:25.198137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:25.198137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:25.198137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:25.198137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:52:25.198137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:25.198137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 00:52:25.198137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:25.202188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 413/835] 10.222.0.42 () {56 vars in 1132 bytes} [Wed Sep 14 00:52:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:25.204032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e1602-b047-44d9-8c1f-646ec2a0c02d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:25.305378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e1602-b047-44d9-8c1f-646ec2a0c02d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:25.340677 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee951ac-1648-4214-a4f7-9e678958bb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:25.341417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee951ac-1648-4214-a4f7-9e678958bb3e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:25.341576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee951ac-1648-4214-a4f7-9e678958bb3e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:25.341803 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee951ac-1648-4214-a4f7-9e678958bb3e None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:25.342433 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee951ac-1648-4214-a4f7-9e678958bb3e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:25.342729 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee951ac-1648-4214-a4f7-9e678958bb3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:25.344335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee951ac-1648-4214-a4f7-9e678958bb3e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:25.349323 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 423/836] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Sep 14 00:52:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:25.354909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b078c2-31a8-49fa-8fe1-e125975c5def None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:25.397121 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:25.480772 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b078c2-31a8-49fa-8fe1-e125975c5def None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:25.518308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533461b-2515-49b6-b29b-9d0041dd87b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:25.518919 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533461b-2515-49b6-b29b-9d0041dd87b6 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:25.519195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533461b-2515-49b6-b29b-9d0041dd87b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:25.519466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533461b-2515-49b6-b29b-9d0041dd87b6 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:25.520124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533461b-2515-49b6-b29b-9d0041dd87b6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:25.520427 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533461b-2515-49b6-b29b-9d0041dd87b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:25.521788 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533461b-2515-49b6-b29b-9d0041dd87b6 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:25.531753 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 414/837] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Sep 14 00:52:25 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:25.566225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5d288-8a01-4f4a-a84a-13fb06d07dcf None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:25.566528 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5d288-8a01-4f4a-a84a-13fb06d07dcf None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:25.566698 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5d288-8a01-4f4a-a84a-13fb06d07dcf None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:25.568447 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 424/838] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:25.574098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99dbf5-e655-4ec2-8cc1-aa51156f9042 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:25.574394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99dbf5-e655-4ec2-8cc1-aa51156f9042 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:25.574675 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99dbf5-e655-4ec2-8cc1-aa51156f9042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:25.594929 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-1b99dbf5-e655-4ec2-8cc1-aa51156f9042 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:25.596242 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1b99dbf5-e655-4ec2-8cc1-aa51156f9042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:25.672814 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 415/839] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:25.943326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5d288-8a01-4f4a-a84a-13fb06d07dcf None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:25.972390 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:26.052789 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5d288-8a01-4f4a-a84a-13fb06d07dcf None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:26.087515 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66b921-da62-4c01-9fc1-92c873e133c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:26.088181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66b921-da62-4c01-9fc1-92c873e133c7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:26.088456 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66b921-da62-4c01-9fc1-92c873e133c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:26.088724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66b921-da62-4c01-9fc1-92c873e133c7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:26.089301 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66b921-da62-4c01-9fc1-92c873e133c7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:26.090772 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66b921-da62-4c01-9fc1-92c873e133c7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:26.101153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-0c66b921-da62-4c01-9fc1-92c873e133c7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 00:52:26.101153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:26.101153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:52:26.101153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:52:26.101153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:52:26.101153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:52:26.101153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:52:26.101153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:26.101153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 00:52:26.101153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:26.101153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:26.101153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:26.101153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:26.101153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:26.101153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:26.101153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:26.101153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:26.102251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:26.102251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:26.102251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:26.102251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:26.102251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:26.102251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:26.102251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:52:26.102251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:52:26.102251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:52:26.102251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:52:26.102251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:26.102251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:26.102251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:26.102251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:26.102251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:26.102251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:26.103107 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:26.103107 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:26.103107 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:26.103107 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:26.103107 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:26.103107 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:26.103107 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:26.103107 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:26.103107 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:26.103107 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:26.103107 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:52:26.103107 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:26.103107 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 00:52:26.103107 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:26.103107 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 425/840] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 00:52:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:26.108312 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b99dbf5-e655-4ec2-8cc1-aa51156f9042 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:26.139315 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:26.339611 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b99dbf5-e655-4ec2-8cc1-aa51156f9042 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:26.375047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4113e-d3ea-4185-af86-c621967f3d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:26.375724 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4113e-d3ea-4185-af86-c621967f3d8b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:26.376002 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4113e-d3ea-4185-af86-c621967f3d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:26.376273 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4113e-d3ea-4185-af86-c621967f3d8b None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:26.376905 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef4113e-d3ea-4185-af86-c621967f3d8b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:26.377201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef4113e-d3ea-4185-af86-c621967f3d8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:26.378620 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef4113e-d3ea-4185-af86-c621967f3d8b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:26.383483 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 416/841] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 00:52:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:26.388790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66b921-da62-4c01-9fc1-92c873e133c7 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:26.423867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685cf6b-c5dc-4e32-b69c-1367c489af76 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:26.424212 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685cf6b-c5dc-4e32-b69c-1367c489af76 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:26.424422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685cf6b-c5dc-4e32-b69c-1367c489af76 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:26.425985 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 417/842] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:52:26.430744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be4846-de5a-4ccf-b0d9-3fc1c84366b3 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:26.431023 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be4846-de5a-4ccf-b0d9-3fc1c84366b3 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:26.431295 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be4846-de5a-4ccf-b0d9-3fc1c84366b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:26.454524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-54be4846-de5a-4ccf-b0d9-3fc1c84366b3 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:26.456385 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-54be4846-de5a-4ccf-b0d9-3fc1c84366b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:26.503309 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:26.514696 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66b921-da62-4c01-9fc1-92c873e133c7 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:26.558832 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 418/843] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 00:52:26.559048 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66bdb64-5c47-440c-b267-2240b2a66355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:26.559048 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66bdb64-5c47-440c-b267-2240b2a66355 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:26.559048 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66bdb64-5c47-440c-b267-2240b2a66355 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:26.559048 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66bdb64-5c47-440c-b267-2240b2a66355 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:26.559337 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66bdb64-5c47-440c-b267-2240b2a66355 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:26.559337 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66bdb64-5c47-440c-b267-2240b2a66355 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:26.561673 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66bdb64-5c47-440c-b267-2240b2a66355 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:26.568172 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 426/844] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 00:52:26 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:26.752819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a94a28-1207-4f1a-81d2-88be5d6c35ba None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:26.753359 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a94a28-1207-4f1a-81d2-88be5d6c35ba None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:26.753359 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a94a28-1207-4f1a-81d2-88be5d6c35ba None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:26.754931 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 419/845] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:26.760269 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcdbfe-0645-40e3-90c6-6faeff44aa38 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:26.760559 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcdbfe-0645-40e3-90c6-6faeff44aa38 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:26.760838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcdbfe-0645-40e3-90c6-6faeff44aa38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:26.780400 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-47fcdbfe-0645-40e3-90c6-6faeff44aa38 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:26.782140 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-47fcdbfe-0645-40e3-90c6-6faeff44aa38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:26.839892 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a94a28-1207-4f1a-81d2-88be5d6c35ba None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:26.852651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 427/846] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:26.870380 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:26.958914 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a94a28-1207-4f1a-81d2-88be5d6c35ba None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:26.994483 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d957e8f-480a-4068-8c9e-7b1442d09732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:26.995120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d957e8f-480a-4068-8c9e-7b1442d09732 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:26.995399 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d957e8f-480a-4068-8c9e-7b1442d09732 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:26.995699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d957e8f-480a-4068-8c9e-7b1442d09732 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:27.006680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d957e8f-480a-4068-8c9e-7b1442d09732 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:52:27.011345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-7d957e8f-480a-4068-8c9e-7b1442d09732 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:52:27.011345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:27.011345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:27.011345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:27.011345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:27.011345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:27.011345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:27.011345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:27.011345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:27.011345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:27.011345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:27.011345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:27.011345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:52:27.011345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:52:27.011345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:52:27.011345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:52:27.012430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:52:27.012430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:52:27.012430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:52:27.012430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:52:27.012430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:27.012430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:27.012430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:27.012430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:27.012430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:27.012430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:27.012430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:27.012430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:27.012430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:27.012430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:27.012430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:27.012430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:27.013532 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:27.013532 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:27.013532 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:27.013532 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:27.013532 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:52:27.013532 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:52:27.013532 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:52:27.013532 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:52:27.013532 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:52:27.013532 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:52:27.013532 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:52:27.013532 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:27.013532 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 420/847] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 00:52:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:27.021671 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fcdbfe-0645-40e3-90c6-6faeff44aa38 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:27.112687 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:27.130599 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fcdbfe-0645-40e3-90c6-6faeff44aa38 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:27.152255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d957e8f-480a-4068-8c9e-7b1442d09732 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:27.170296 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b397d-4f6b-46cc-880f-b7c1f4ade1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:27.172051 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b397d-4f6b-46cc-880f-b7c1f4ade1a5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:27.172575 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b397d-4f6b-46cc-880f-b7c1f4ade1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:27.173103 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b397d-4f6b-46cc-880f-b7c1f4ade1a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:27.173975 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b397d-4f6b-46cc-880f-b7c1f4ade1a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:27.175078 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b397d-4f6b-46cc-880f-b7c1f4ade1a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:27.176973 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b397d-4f6b-46cc-880f-b7c1f4ade1a5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:27.191464 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 428/848] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 00:52:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:27.193003 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b397d-4f6b-46cc-880f-b7c1f4ade1a5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:27.240772 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:27.262436 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d957e8f-480a-4068-8c9e-7b1442d09732 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:27.301061 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145ca0a-d0c2-445c-9ffd-ff45dd918290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:27.301700 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145ca0a-d0c2-445c-9ffd-ff45dd918290 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:27.301985 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145ca0a-d0c2-445c-9ffd-ff45dd918290 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:27.302306 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145ca0a-d0c2-445c-9ffd-ff45dd918290 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:27.314757 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8145ca0a-d0c2-445c-9ffd-ff45dd918290 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:52:27.319459 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b397d-4f6b-46cc-880f-b7c1f4ade1a5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:27.319876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-8145ca0a-d0c2-445c-9ffd-ff45dd918290 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:52:27.319876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:27.319876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:27.319876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:27.319876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:27.319876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:27.319876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:27.319876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:27.319876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:27.319876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:27.319876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:27.319876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:27.319876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:52:27.319876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:52:27.319876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:52:27.319876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:52:27.320780 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:52:27.320780 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:52:27.320780 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:52:27.320780 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:52:27.320780 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:27.320780 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:27.320780 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:27.320780 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:27.320780 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:27.320780 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:27.320780 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:27.320780 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:27.320780 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:27.320780 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:27.320780 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:27.320780 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:27.321640 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:27.321640 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:27.321640 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:27.321640 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:27.321640 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:52:27.321640 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:52:27.321640 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:52:27.321640 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:52:27.321640 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:52:27.321640 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:52:27.321640 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:52:27.321640 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:27.326794 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 421/849] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:52:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:52:27.328050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145ca0a-d0c2-445c-9ffd-ff45dd918290 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:27.367573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c69453-1c91-456d-aff2-332a5479555f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:27.369437 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c69453-1c91-456d-aff2-332a5479555f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:27.369919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c69453-1c91-456d-aff2-332a5479555f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:27.370376 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c69453-1c91-456d-aff2-332a5479555f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:27.385100 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c69453-1c91-456d-aff2-332a5479555f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:27.387183 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c69453-1c91-456d-aff2-332a5479555f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:27.396922 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:27.408345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-67c69453-1c91-456d-aff2-332a5479555f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:27.408345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:27.408345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:27.408345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:27.408345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:27.408345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:27.408345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:27.408345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:27.408345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:27.408345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:27.408345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:27.408345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:27.408345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:27.408345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:27.408345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:27.408345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:27.409537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:27.409537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:27.409537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:27.409537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:27.409537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:27.409537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:27.409537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:27.409537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:27.409537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:27.409537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:27.409537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:27.409537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:27.409537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:27.409537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:27.409537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:27.409537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:27.409537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:27.410675 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:27.410675 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:27.410675 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:27.416215 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 429/850] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:52:27.417691 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c69453-1c91-456d-aff2-332a5479555f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:27.451959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145ca0a-d0c2-445c-9ffd-ff45dd918290 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:27.504415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91b839-ffe2-495b-b9a7-4d8456de1e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:27.505549 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91b839-ffe2-495b-b9a7-4d8456de1e7e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:27.505898 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91b839-ffe2-495b-b9a7-4d8456de1e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:27.506261 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91b839-ffe2-495b-b9a7-4d8456de1e7e None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:27.507158 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91b839-ffe2-495b-b9a7-4d8456de1e7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:27.507530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91b839-ffe2-495b-b9a7-4d8456de1e7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:27.509913 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91b839-ffe2-495b-b9a7-4d8456de1e7e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:27.523697 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 422/851] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:52:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:27.528119 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91b839-ffe2-495b-b9a7-4d8456de1e7e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:27.532605 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:27.563101 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c69453-1c91-456d-aff2-332a5479555f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:27.598557 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b83cd1-9c82-435e-9f09-25b478925e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:27.599340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b83cd1-9c82-435e-9f09-25b478925e93 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:27.599711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b83cd1-9c82-435e-9f09-25b478925e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:27.600057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b83cd1-9c82-435e-9f09-25b478925e93 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:27.600772 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b83cd1-9c82-435e-9f09-25b478925e93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:27.601144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b83cd1-9c82-435e-9f09-25b478925e93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:27.602650 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b83cd1-9c82-435e-9f09-25b478925e93 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:27.620488 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 430/852] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:27.622754 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b83cd1-9c82-435e-9f09-25b478925e93 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:27.649723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91b839-ffe2-495b-b9a7-4d8456de1e7e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:27.695873 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c879890-b2b7-473a-89e6-2ba576c08e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:27.696790 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c879890-b2b7-473a-89e6-2ba576c08e80 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:27.697312 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c879890-b2b7-473a-89e6-2ba576c08e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:27.697738 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c879890-b2b7-473a-89e6-2ba576c08e80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:27.698492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:27.712113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c879890-b2b7-473a-89e6-2ba576c08e80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:27.714352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c879890-b2b7-473a-89e6-2ba576c08e80 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:27.732693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-4c879890-b2b7-473a-89e6-2ba576c08e80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:27.732693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:27.732693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:27.732693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:27.732693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:27.732693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:27.732693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:27.732693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:27.732693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:27.732693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:27.732693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:27.732693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:27.732693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:27.732693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:27.732693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:27.732693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:27.734502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:27.734502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:27.734502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:27.734502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:27.734502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:27.734502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:27.734502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:27.734502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:27.734502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:27.734502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:27.734502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:27.734502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:27.734502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:27.734502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:27.734502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:27.734502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:27.734502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:27.735734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:27.735734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:27.735734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:27.740060 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 423/853] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:27.741214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c879890-b2b7-473a-89e6-2ba576c08e80 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:27.765979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b83cd1-9c82-435e-9f09-25b478925e93 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:27.816168 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:27.822061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c8a93-24d8-40a9-9787-b8175ca07161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:27.822996 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c8a93-24d8-40a9-9787-b8175ca07161 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:27.823447 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c8a93-24d8-40a9-9787-b8175ca07161 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:27.823895 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c8a93-24d8-40a9-9787-b8175ca07161 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:27.832925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011c8a93-24d8-40a9-9787-b8175ca07161 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:27.834810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011c8a93-24d8-40a9-9787-b8175ca07161 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:27.848513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-011c8a93-24d8-40a9-9787-b8175ca07161 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 00:52:27.848513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:27.848513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:27.848513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:27.848513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:27.848513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:27.848513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:27.848513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:27.848513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:27.848513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:27.848513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:27.848513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:27.848513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:52:27.848513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:52:27.848513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:52:27.848513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:52:27.849695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:27.849695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:27.849695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:52:27.849695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:27.849695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:52:27.849695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:27.849695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:27.849695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:27.849695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:27.849695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:27.849695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:27.849695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:27.849695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:27.849695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:27.849695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:27.849695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:27.849695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:27.849695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:27.850877 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:27.850877 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:27.850877 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:52:27.850877 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:52:27.850877 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:52:27.850877 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:52:27.850877 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:52:27.850877 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:52:27.850877 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 00:52:27.850877 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:27.853033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 431/854] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Sep 14 00:52:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:27.860958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c8a93-24d8-40a9-9787-b8175ca07161 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:27.874377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c879890-b2b7-473a-89e6-2ba576c08e80 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:27.910128 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb08eed-68ff-4fdb-88f2-47ff294fc554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:27.910771 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb08eed-68ff-4fdb-88f2-47ff294fc554 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:27.911027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb08eed-68ff-4fdb-88f2-47ff294fc554 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:27.911300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb08eed-68ff-4fdb-88f2-47ff294fc554 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:27.912049 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb08eed-68ff-4fdb-88f2-47ff294fc554 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:27.912276 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb08eed-68ff-4fdb-88f2-47ff294fc554 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:27.913649 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb08eed-68ff-4fdb-88f2-47ff294fc554 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:27.930489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 424/855] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:27.930489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9b5d7-e250-4e71-9e68-46cc54f76d47 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:27.930489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9b5d7-e250-4e71-9e68-46cc54f76d47 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:27.930489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9b5d7-e250-4e71-9e68-46cc54f76d47 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:27.930489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 425/856] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:27.933423 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9b5d7-e250-4e71-9e68-46cc54f76d47 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:27.976605 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c8a93-24d8-40a9-9787-b8175ca07161 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:27.994041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:28.013892 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24685a1-6fc1-4c4d-84ae-0d541713dfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:28.014504 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24685a1-6fc1-4c4d-84ae-0d541713dfc2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:28.014794 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24685a1-6fc1-4c4d-84ae-0d541713dfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:28.015250 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24685a1-6fc1-4c4d-84ae-0d541713dfc2 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:28.016274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24685a1-6fc1-4c4d-84ae-0d541713dfc2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:28.016585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24685a1-6fc1-4c4d-84ae-0d541713dfc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:28.018014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24685a1-6fc1-4c4d-84ae-0d541713dfc2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:28.036162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 432/857] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Sep 14 00:52:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:28.037239 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c97c4b-1cfb-4fac-9432-eb0bb84adcf4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:28.037525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c97c4b-1cfb-4fac-9432-eb0bb84adcf4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:28.037796 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c97c4b-1cfb-4fac-9432-eb0bb84adcf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:28.050712 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9b5d7-e250-4e71-9e68-46cc54f76d47 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:28.056374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-16c97c4b-1cfb-4fac-9432-eb0bb84adcf4 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:28.057545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-16c97c4b-1cfb-4fac-9432-eb0bb84adcf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:28.069570 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:28.088800 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b1e42-a20a-4e99-a5a7-c8e9ecd98449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:28.089438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b1e42-a20a-4e99-a5a7-c8e9ecd98449 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:28.089836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b1e42-a20a-4e99-a5a7-c8e9ecd98449 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:28.090181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b1e42-a20a-4e99-a5a7-c8e9ecd98449 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:28.098197 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b1e42-a20a-4e99-a5a7-c8e9ecd98449 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:28.099674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b1e42-a20a-4e99-a5a7-c8e9ecd98449 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:28.112235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-072b1e42-a20a-4e99-a5a7-c8e9ecd98449 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 00:52:28.112235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:28.112235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:28.112235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:28.112235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:28.112235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:28.112235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:28.112235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:28.112235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:28.112235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:28.112235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:28.112235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:28.112235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:52:28.112235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:52:28.112235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:52:28.112235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:52:28.113172 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:28.113172 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:28.113172 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:52:28.113172 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:28.113172 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:52:28.113172 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:28.113172 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:28.113172 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:28.113172 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:28.113172 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:28.113172 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:28.113172 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:28.113172 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:28.113172 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:28.113172 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:28.113172 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:28.113172 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:28.113172 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:28.114152 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:28.114152 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:28.114152 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:52:28.114152 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:52:28.114152 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:52:28.114152 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:52:28.114152 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:52:28.114152 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:52:28.114152 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 00:52:28.114152 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:28.118170 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 426/858] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:52:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:28.119112 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b1e42-a20a-4e99-a5a7-c8e9ecd98449 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:28.139723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 433/859] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:28.139723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c97c4b-1cfb-4fac-9432-eb0bb84adcf4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:28.257331 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c97c4b-1cfb-4fac-9432-eb0bb84adcf4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:28.258181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b1e42-a20a-4e99-a5a7-c8e9ecd98449 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:28.268011 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:28.303399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be0c65-accc-421f-97ea-72f2b1227251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:28.304049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be0c65-accc-421f-97ea-72f2b1227251 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:28.304449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be0c65-accc-421f-97ea-72f2b1227251 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:28.304599 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89262a3-a6f5-42e9-9d02-9b1ba96ab118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:28.304717 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be0c65-accc-421f-97ea-72f2b1227251 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:28.305396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89262a3-a6f5-42e9-9d02-9b1ba96ab118 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:28.305509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7be0c65-accc-421f-97ea-72f2b1227251 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:28.305814 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7be0c65-accc-421f-97ea-72f2b1227251 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:28.305916 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89262a3-a6f5-42e9-9d02-9b1ba96ab118 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:28.306259 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89262a3-a6f5-42e9-9d02-9b1ba96ab118 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:28.307377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7be0c65-accc-421f-97ea-72f2b1227251 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:28.335803 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 434/860] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Sep 14 00:52:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:28.337220 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be0c65-accc-421f-97ea-72f2b1227251 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:28.368467 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:28.399668 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89262a3-a6f5-42e9-9d02-9b1ba96ab118 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:28.401259 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89262a3-a6f5-42e9-9d02-9b1ba96ab118 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:28.455862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be0c65-accc-421f-97ea-72f2b1227251 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:28.499321 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce54e2-d7f5-4293-800d-b1f55fe10210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:28.500431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce54e2-d7f5-4293-800d-b1f55fe10210 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:28.501044 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce54e2-d7f5-4293-800d-b1f55fe10210 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:28.501321 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce54e2-d7f5-4293-800d-b1f55fe10210 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:28.511603 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 427/861] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:52:28 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 00:52:28.511603 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89262a3-a6f5-42e9-9d02-9b1ba96ab118 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:28.538859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:28.579401 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce54e2-d7f5-4293-800d-b1f55fe10210 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:28.580881 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce54e2-d7f5-4293-800d-b1f55fe10210 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:28.595415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:28.620017 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89262a3-a6f5-42e9-9d02-9b1ba96ab118 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:28.655161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361fef5-4cb6-444a-bc0b-25170de70086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:28.655944 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361fef5-4cb6-444a-bc0b-25170de70086 None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:28.656222 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361fef5-4cb6-444a-bc0b-25170de70086 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:28.656532 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361fef5-4cb6-444a-bc0b-25170de70086 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:28.657210 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6361fef5-4cb6-444a-bc0b-25170de70086 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:28.658624 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6361fef5-4cb6-444a-bc0b-25170de70086 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:28.675503 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 428/862] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 00:52:28 2022] POST /identity/v3/services => generated 269 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:52:28.676803 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361fef5-4cb6-444a-bc0b-25170de70086 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:28.701508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 435/863] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:52:28 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 00:52:28.708317 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce54e2-d7f5-4293-800d-b1f55fe10210 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:28.795268 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:28.819202 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361fef5-4cb6-444a-bc0b-25170de70086 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:28.839990 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce54e2-d7f5-4293-800d-b1f55fe10210 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:28.852225 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:28.857882 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923c9b7-785e-4be9-a71e-cafe33885297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:28.859281 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923c9b7-785e-4be9-a71e-cafe33885297 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:28.859676 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923c9b7-785e-4be9-a71e-cafe33885297 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:28.859936 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923c9b7-785e-4be9-a71e-cafe33885297 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:28.873378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3923c9b7-785e-4be9-a71e-cafe33885297 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:28.875822 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3923c9b7-785e-4be9-a71e-cafe33885297 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:28.886751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cbc545-01c8-417a-bf86-a111aa4a5321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:28.887389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbc545-01c8-417a-bf86-a111aa4a5321 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:28.887695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbc545-01c8-417a-bf86-a111aa4a5321 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:28.887969 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbc545-01c8-417a-bf86-a111aa4a5321 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:28.890845 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-3923c9b7-785e-4be9-a71e-cafe33885297 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:28.890845 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:28.890845 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:28.890845 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:28.890845 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:28.890845 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:28.890845 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:28.890845 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:28.890845 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:28.890845 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:28.890845 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:28.890845 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:28.890845 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:28.890845 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:28.890845 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:28.890845 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:28.891842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:28.891842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:28.891842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:28.891842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:28.891842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:28.891842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:28.891842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:28.891842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:28.891842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:28.891842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:28.891842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:28.891842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:28.891842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:28.891842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:28.891842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:28.891842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:28.891842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:28.892748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:28.892748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:28.892748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:28.892748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 429/864] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:28.893166 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2a1ab-0c7b-4b1e-bf55-74f3c0570a6a None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:28.893444 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2a1ab-0c7b-4b1e-bf55-74f3c0570a6a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:28.893695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2a1ab-0c7b-4b1e-bf55-74f3c0570a6a None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:28.894916 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 430/865] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:28.899793 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2a1ab-0c7b-4b1e-bf55-74f3c0570a6a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:28.902137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cbc545-01c8-417a-bf86-a111aa4a5321 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:28.903739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cbc545-01c8-417a-bf86-a111aa4a5321 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:28.919265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-77cbc545-01c8-417a-bf86-a111aa4a5321 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:28.919265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:28.919265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:28.919265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:28.919265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:28.919265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:28.919265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:28.919265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:28.919265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:28.919265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:28.919265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:28.919265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:28.919265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:28.919265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:28.919265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:28.919265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:28.920424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:28.920424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:28.920424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:28.920424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:28.920424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:28.920424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:28.920424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:28.920424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:28.920424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:28.920424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:28.920424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:28.920424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:28.920424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:28.920424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:28.920424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:28.920424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:28.920424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:28.921624 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:28.921624 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:28.921624 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:28.921624 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 436/866] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:28.925515 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4370038-a6ca-46a2-a3d1-f2327e2c39be None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:28.925515 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4370038-a6ca-46a2-a3d1-f2327e2c39be None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:28.925515 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4370038-a6ca-46a2-a3d1-f2327e2c39be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:28.941747 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d4370038-a6ca-46a2-a3d1-f2327e2c39be None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:28.942919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d4370038-a6ca-46a2-a3d1-f2327e2c39be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:29.013940 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 437/867] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:28 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:29.014571 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2a1ab-0c7b-4b1e-bf55-74f3c0570a6a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:29.014777 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4370038-a6ca-46a2-a3d1-f2327e2c39be None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:29.067271 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f57aee-5cfb-4a9e-892c-49b6050421de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:29.068324 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f57aee-5cfb-4a9e-892c-49b6050421de None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:29.069252 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f57aee-5cfb-4a9e-892c-49b6050421de None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:29.069738 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f57aee-5cfb-4a9e-892c-49b6050421de None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:29.071198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f57aee-5cfb-4a9e-892c-49b6050421de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:29.071822 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f57aee-5cfb-4a9e-892c-49b6050421de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:29.073685 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f57aee-5cfb-4a9e-892c-49b6050421de None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:29.074529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:29.093648 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 431/868] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:29.098941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f57aee-5cfb-4a9e-892c-49b6050421de None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:29.143401 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4370038-a6ca-46a2-a3d1-f2327e2c39be None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:29.156935 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:29.185117 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0239d-fef3-48aa-acbc-b5dc5573fc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:29.185731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0239d-fef3-48aa-acbc-b5dc5573fc44 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:29.186013 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0239d-fef3-48aa-acbc-b5dc5573fc44 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:29.186295 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0239d-fef3-48aa-acbc-b5dc5573fc44 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:29.187489 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f0239d-fef3-48aa-acbc-b5dc5573fc44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:29.187909 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f0239d-fef3-48aa-acbc-b5dc5573fc44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:29.189661 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f0239d-fef3-48aa-acbc-b5dc5573fc44 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:29.201705 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 438/869] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:29.206989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0239d-fef3-48aa-acbc-b5dc5573fc44 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:29.235315 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f57aee-5cfb-4a9e-892c-49b6050421de None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:29.284984 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5534720-01c8-4d8f-9903-11b5a56488f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:29.285629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5534720-01c8-4d8f-9903-11b5a56488f6 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:29.286515 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5534720-01c8-4d8f-9903-11b5a56488f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:29.286801 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5534720-01c8-4d8f-9903-11b5a56488f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:29.295981 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5534720-01c8-4d8f-9903-11b5a56488f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:29.298109 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5534720-01c8-4d8f-9903-11b5a56488f6 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:29.310617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-e5534720-01c8-4d8f-9903-11b5a56488f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:29.310617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:29.310617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:29.310617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:29.310617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:29.310617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:29.310617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:29.310617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:29.310617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:29.310617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:29.310617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:29.310617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:29.310617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:52:29.310617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:52:29.310617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:52:29.311591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:52:29.311591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:29.311591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:29.311591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:29.311591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:29.311591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:29.311591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:29.311591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:29.311591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:29.311591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:29.311591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:29.311591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:29.311591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:29.311591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:29.311591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:29.311591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:29.311591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:29.312928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:52:29.312928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:52:29.312928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:52:29.312928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:52:29.312928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:52:29.312928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:52:29.312928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:29.312928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:29.312928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 432/870] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:29.312928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5534720-01c8-4d8f-9903-11b5a56488f6 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:29.323704 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0239d-fef3-48aa-acbc-b5dc5573fc44 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:29.332032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:29.361632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1b13d-a1bb-4717-9e68-56cfa43010e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:29.362398 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1b13d-a1bb-4717-9e68-56cfa43010e5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:29.362664 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1b13d-a1bb-4717-9e68-56cfa43010e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:29.362939 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c1b13d-a1bb-4717-9e68-56cfa43010e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:29.370321 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c1b13d-a1bb-4717-9e68-56cfa43010e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:29.371900 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c1b13d-a1bb-4717-9e68-56cfa43010e5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:29.384614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-56c1b13d-a1bb-4717-9e68-56cfa43010e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:29.384614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:29.384614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:29.384614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:29.384614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:29.384614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:29.384614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:29.384614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:29.384614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:29.384614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:29.384614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:29.384614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:29.384614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:52:29.384614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:52:29.384614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:52:29.385509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:52:29.385509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:29.385509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:29.385509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:29.385509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:29.385509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:29.385509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:29.385509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:29.385509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:29.385509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:29.385509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:29.385509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:29.385509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:29.385509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:29.385509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:29.385509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:29.385509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:29.386402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:52:29.386402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:52:29.386402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:52:29.386402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:52:29.386402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:52:29.386402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:52:29.386402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:29.386402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:29.386402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 439/871] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:52:29.386893 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1b13d-a1bb-4717-9e68-56cfa43010e5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:29.421531 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5534720-01c8-4d8f-9903-11b5a56488f6 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:29.423867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:29.457502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb48a7-1872-4ebf-a5e3-36a99567e045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:29.458115 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb48a7-1872-4ebf-a5e3-36a99567e045 None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:29.458396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb48a7-1872-4ebf-a5e3-36a99567e045 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:29.458672 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb48a7-1872-4ebf-a5e3-36a99567e045 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:29.459318 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb48a7-1872-4ebf-a5e3-36a99567e045 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:29.460739 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb48a7-1872-4ebf-a5e3-36a99567e045 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:29.485114 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 433/872] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 00:52:29 2022] POST /identity/v3/services => generated 251 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:52:29.486236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb48a7-1872-4ebf-a5e3-36a99567e045 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:29.508016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c1b13d-a1bb-4717-9e68-56cfa43010e5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:29.569276 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39cfd7-6732-4a84-9959-9fa0ad2197d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:29.569965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39cfd7-6732-4a84-9959-9fa0ad2197d1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:29.570243 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39cfd7-6732-4a84-9959-9fa0ad2197d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:29.570597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39cfd7-6732-4a84-9959-9fa0ad2197d1 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:29.571310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39cfd7-6732-4a84-9959-9fa0ad2197d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:29.571723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39cfd7-6732-4a84-9959-9fa0ad2197d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:29.573523 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39cfd7-6732-4a84-9959-9fa0ad2197d1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:29.596044 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 440/873] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:52:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:29.598111 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39cfd7-6732-4a84-9959-9fa0ad2197d1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:29.632878 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:29.641298 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb48a7-1872-4ebf-a5e3-36a99567e045 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:29.691302 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475f26c-042d-433f-8405-5a53b668c7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:29.691895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475f26c-042d-433f-8405-5a53b668c7d9 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:29.693593 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475f26c-042d-433f-8405-5a53b668c7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:29.693593 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475f26c-042d-433f-8405-5a53b668c7d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:29.693593 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475f26c-042d-433f-8405-5a53b668c7d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:29.693593 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475f26c-042d-433f-8405-5a53b668c7d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:29.694919 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475f26c-042d-433f-8405-5a53b668c7d9 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:29.710521 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 434/874] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:52:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:29.716704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475f26c-042d-433f-8405-5a53b668c7d9 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:29.734162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39cfd7-6732-4a84-9959-9fa0ad2197d1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:29.752052 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:29.781214 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf335a95-555b-48ee-aa6a-a76176db49f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:29.782188 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf335a95-555b-48ee-aa6a-a76176db49f9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:29.782473 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf335a95-555b-48ee-aa6a-a76176db49f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:29.782749 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf335a95-555b-48ee-aa6a-a76176db49f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:29.783891 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf335a95-555b-48ee-aa6a-a76176db49f9 None admin] RBAC: Adding query filter params (role.id=7a33c2ceb92a478f81753573f8ece1bd, scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3, user.id=ceccc909529042539dd2d5015058d8f7) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:29.784212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf335a95-555b-48ee-aa6a-a76176db49f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:29.785649 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf335a95-555b-48ee-aa6a-a76176db49f9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:29.794478 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 441/875] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:52:29 2022] GET /identity/v3/role_assignments?user.id=ceccc909529042539dd2d5015058d8f7&scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3&role.id=7a33c2ceb92a478f81753573f8ece1bd => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:29.845593 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475f26c-042d-433f-8405-5a53b668c7d9 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:29.885325 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e7de0-eb64-4847-973f-395841b03a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:29.885946 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e7de0-eb64-4847-973f-395841b03a32 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:29.886223 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e7de0-eb64-4847-973f-395841b03a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:29.886491 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e7de0-eb64-4847-973f-395841b03a32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:29.887227 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e7de0-eb64-4847-973f-395841b03a32 None admin] RBAC: Adding query filter params (role.id=ada268ea63354a0297070369dfe3c6cc, scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3, user.id=e52776d9b4894af9b85a24136a8255e5) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:29.887533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e7de0-eb64-4847-973f-395841b03a32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:29.889020 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e7de0-eb64-4847-973f-395841b03a32 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:29.896876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 435/876] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:52:29 2022] GET /identity/v3/role_assignments?user.id=e52776d9b4894af9b85a24136a8255e5&scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3&role.id=ada268ea63354a0297070369dfe3c6cc => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:30.939544 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c4b32-d66d-4bdc-aa0b-05b50493e67f None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:30.940173 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c4b32-d66d-4bdc-aa0b-05b50493e67f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:30.940173 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c4b32-d66d-4bdc-aa0b-05b50493e67f None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:30.941668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 442/877] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:30.947086 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6079a6-cb86-4c92-a88f-4d845dbd1a8f None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:30.947500 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6079a6-cb86-4c92-a88f-4d845dbd1a8f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:30.947701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6079a6-cb86-4c92-a88f-4d845dbd1a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:30.965911 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-cf6079a6-cb86-4c92-a88f-4d845dbd1a8f None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:30.967076 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-cf6079a6-cb86-4c92-a88f-4d845dbd1a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:31.040913 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 436/878] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:30 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 00:52:31.315292 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c4b32-d66d-4bdc-aa0b-05b50493e67f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:31.405162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:31.423367 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c4b32-d66d-4bdc-aa0b-05b50493e67f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:31.457748 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff05d7-aafa-4d49-aceb-11bcd3448ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:31.458420 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff05d7-aafa-4d49-aceb-11bcd3448ece None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:31.458647 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff05d7-aafa-4d49-aceb-11bcd3448ece None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:31.458919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff05d7-aafa-4d49-aceb-11bcd3448ece None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:31.459490 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff05d7-aafa-4d49-aceb-11bcd3448ece None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:31.460882 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff05d7-aafa-4d49-aceb-11bcd3448ece None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:31.475470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-cbff05d7-aafa-4d49-aceb-11bcd3448ece None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 00:52:31.475470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:31.475470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:52:31.475470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:52:31.475470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:52:31.475470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:52:31.475470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:52:31.475470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:31.475470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 00:52:31.475470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:31.475470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:31.475470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:31.475470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:31.475470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:31.475470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:31.475470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:31.476545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:31.476545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:31.476545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:31.476545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:31.476545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:31.476545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:31.476545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:31.476545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:52:31.476545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:52:31.476545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:52:31.476545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:52:31.476545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:31.476545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:31.476545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:31.476545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:31.476545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:31.476545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:31.477537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:31.477537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:31.477537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:31.477537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:31.477537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:31.477537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:31.477537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:31.477537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:31.477537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:31.477537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:31.477537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:52:31.477537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:31.477537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 00:52:31.477537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:31.487233 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 443/879] 10.222.0.42 () {56 vars in 1132 bytes} [Wed Sep 14 00:52:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:31.488614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6079a6-cb86-4c92-a88f-4d845dbd1a8f None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:31.507765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:31.591648 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6079a6-cb86-4c92-a88f-4d845dbd1a8f None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:31.628581 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5947f-3249-4f34-85dd-e9ac4708453b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:31.629190 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5947f-3249-4f34-85dd-e9ac4708453b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:31.629464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5947f-3249-4f34-85dd-e9ac4708453b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:31.629732 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5947f-3249-4f34-85dd-e9ac4708453b None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:31.630356 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5947f-3249-4f34-85dd-e9ac4708453b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:31.630657 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5947f-3249-4f34-85dd-e9ac4708453b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:31.632051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5947f-3249-4f34-85dd-e9ac4708453b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:31.642268 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 437/880] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Sep 14 00:52:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:31.643714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff05d7-aafa-4d49-aceb-11bcd3448ece None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:31.758818 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff05d7-aafa-4d49-aceb-11bcd3448ece None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:31.781061 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6aa6c-7054-4f3c-9b83-5fa05f3b683f None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:31.781500 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6aa6c-7054-4f3c-9b83-5fa05f3b683f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:31.781885 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6aa6c-7054-4f3c-9b83-5fa05f3b683f None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:31.783532 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 438/881] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:52:31.788599 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247849ff-62ca-4d7c-8044-0c9e774d1c45 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:31.788965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247849ff-62ca-4d7c-8044-0c9e774d1c45 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:31.789331 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247849ff-62ca-4d7c-8044-0c9e774d1c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:31.799829 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031cd5cd-4c6b-4e0f-8013-ba3e9f2fa19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:31.800709 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cd5cd-4c6b-4e0f-8013-ba3e9f2fa19e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:31.801208 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cd5cd-4c6b-4e0f-8013-ba3e9f2fa19e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:31.801765 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cd5cd-4c6b-4e0f-8013-ba3e9f2fa19e None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:31.802589 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031cd5cd-4c6b-4e0f-8013-ba3e9f2fa19e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:31.803074 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031cd5cd-4c6b-4e0f-8013-ba3e9f2fa19e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:31.804659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031cd5cd-4c6b-4e0f-8013-ba3e9f2fa19e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:31.808181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-247849ff-62ca-4d7c-8044-0c9e774d1c45 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:31.809480 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-247849ff-62ca-4d7c-8044-0c9e774d1c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:31.810863 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 444/882] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Sep 14 00:52:31 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:31.817246 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031cd5cd-4c6b-4e0f-8013-ba3e9f2fa19e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:31.844434 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:31.887811 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 439/883] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:31.929334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031cd5cd-4c6b-4e0f-8013-ba3e9f2fa19e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:31.965664 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d50f60-4036-405a-9071-660fef2a8a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:31.966998 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d50f60-4036-405a-9071-660fef2a8a98 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:31.967437 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d50f60-4036-405a-9071-660fef2a8a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:31.967907 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d50f60-4036-405a-9071-660fef2a8a98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:31.968777 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d50f60-4036-405a-9071-660fef2a8a98 None admin] RBAC: Adding query filter params (interface=public, service_id=644da98dc9084cf887556bc015c6a66d, region_id=RegionOne) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:31.969253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d50f60-4036-405a-9071-660fef2a8a98 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:31.970838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d50f60-4036-405a-9071-660fef2a8a98 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:31.979124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 445/884] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Sep 14 00:52:31 2022] GET /identity/v3/endpoints?service_id=644da98dc9084cf887556bc015c6a66d&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:31.985157 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247849ff-62ca-4d7c-8044-0c9e774d1c45 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:32.000729 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:32.099792 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247849ff-62ca-4d7c-8044-0c9e774d1c45 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:32.145439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ab56b-f910-415f-bf21-cf9602179178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:32.146068 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ab56b-f910-415f-bf21-cf9602179178 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:32.146343 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ab56b-f910-415f-bf21-cf9602179178 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:32.146612 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ab56b-f910-415f-bf21-cf9602179178 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:32.147222 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ab56b-f910-415f-bf21-cf9602179178 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:32.148659 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ab56b-f910-415f-bf21-cf9602179178 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:32.151805 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2e893-3d4f-4090-a836-f0f65073ed4b None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:32.152092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2e893-3d4f-4090-a836-f0f65073ed4b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:32.152359 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2e893-3d4f-4090-a836-f0f65073ed4b None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:32.153964 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 446/885] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:32.154946 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 440/886] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Sep 14 00:52:31 2022] GET /identity/v3/services => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 00:52:32.161753 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ef66b-2875-4c19-bee9-fe2c40892a03 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:32.162036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ef66b-2875-4c19-bee9-fe2c40892a03 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:32.162304 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ef66b-2875-4c19-bee9-fe2c40892a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:32.183002 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b22ef66b-2875-4c19-bee9-fe2c40892a03 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:32.184266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b22ef66b-2875-4c19-bee9-fe2c40892a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:32.191624 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ab56b-f910-415f-bf21-cf9602179178 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:32.272658 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 447/887] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:32 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:32.274312 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080713a3-ff02-45bd-9d3b-69546e10b44b None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:32.274794 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080713a3-ff02-45bd-9d3b-69546e10b44b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:32.275273 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080713a3-ff02-45bd-9d3b-69546e10b44b None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:32.277052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 448/888] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:32.282492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88dc1aa-bfcd-4d4b-9d0e-72bca18aa97f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:32.283061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88dc1aa-bfcd-4d4b-9d0e-72bca18aa97f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:32.283515 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88dc1aa-bfcd-4d4b-9d0e-72bca18aa97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:32.288078 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:32.306947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d88dc1aa-bfcd-4d4b-9d0e-72bca18aa97f None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:32.308661 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d88dc1aa-bfcd-4d4b-9d0e-72bca18aa97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:32.311457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:32.319013 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ab56b-f910-415f-bf21-cf9602179178 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:32.368412 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba768f8-c683-4dc5-8862-8c61bf830156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:32.368987 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba768f8-c683-4dc5-8862-8c61bf830156 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:32.369265 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba768f8-c683-4dc5-8862-8c61bf830156 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:32.369540 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba768f8-c683-4dc5-8862-8c61bf830156 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:32.370136 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba768f8-c683-4dc5-8862-8c61bf830156 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:32.372499 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba768f8-c683-4dc5-8862-8c61bf830156 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:32.381095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-1ba768f8-c683-4dc5-8862-8c61bf830156 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 00:52:32.381095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:32.381095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:52:32.381095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:52:32.381095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:52:32.381095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:52:32.381095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:52:32.381095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:32.381095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 00:52:32.381095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:32.381095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:32.381095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:32.381095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:32.381095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:32.381095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:32.381095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:32.381095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:32.382460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:32.382460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:32.382460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:32.382460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:32.382460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:32.382460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:32.382460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:52:32.382460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:52:32.382460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:52:32.382460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:52:32.382460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:32.382460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:32.382460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:32.382460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:32.382460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:32.382460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:32.383748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:32.383748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:32.383748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:32.383748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:32.383748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:32.383748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:32.383748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:32.383748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:32.383748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:32.383748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:32.383748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:52:32.383748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:32.383748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 00:52:32.383748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:32.383748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 441/889] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 00:52:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:52:32.388019 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba768f8-c683-4dc5-8862-8c61bf830156 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:32.405081 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 449/890] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:32 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 00:52:32.514456 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba768f8-c683-4dc5-8862-8c61bf830156 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:32.550581 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55100cd-a850-4f2a-b20d-0e26ccb99e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:32.551199 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55100cd-a850-4f2a-b20d-0e26ccb99e01 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:32.551476 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55100cd-a850-4f2a-b20d-0e26ccb99e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:32.551779 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55100cd-a850-4f2a-b20d-0e26ccb99e01 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:32.552461 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55100cd-a850-4f2a-b20d-0e26ccb99e01 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:32.552764 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55100cd-a850-4f2a-b20d-0e26ccb99e01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:32.554132 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55100cd-a850-4f2a-b20d-0e26ccb99e01 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:32.558696 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88dc1aa-bfcd-4d4b-9d0e-72bca18aa97f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:32.559459 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 442/891] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 00:52:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:32.567274 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55100cd-a850-4f2a-b20d-0e26ccb99e01 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:32.570897 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:32.682248 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88dc1aa-bfcd-4d4b-9d0e-72bca18aa97f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:32.698817 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55100cd-a850-4f2a-b20d-0e26ccb99e01 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:32.719406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd7ee9-bb40-43ac-bb41-f400b73d730b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:32.720076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd7ee9-bb40-43ac-bb41-f400b73d730b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:32.720361 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd7ee9-bb40-43ac-bb41-f400b73d730b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:32.720635 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd7ee9-bb40-43ac-bb41-f400b73d730b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:32.721221 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd7ee9-bb40-43ac-bb41-f400b73d730b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:32.722605 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd7ee9-bb40-43ac-bb41-f400b73d730b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:32.732001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-56fd7ee9-bb40-43ac-bb41-f400b73d730b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 00:52:32.732001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:32.732001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:52:32.732001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:52:32.732001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:52:32.732001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:52:32.732001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:52:32.732001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:32.732001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 00:52:32.732001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:32.732001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:32.732001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:32.732001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:32.732001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:32.732001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:32.732001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:32.732001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:32.732989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:32.732989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:32.732989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:32.732989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:32.732989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:32.732989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:32.732989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:52:32.732989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:52:32.732989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:52:32.732989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:52:32.732989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:32.732989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:32.732989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:32.732989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:32.732989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:32.732989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:32.734587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:32.734587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:32.734587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:32.734587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:32.734587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:32.734587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:32.734587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:32.734587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:32.734587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:32.734587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:32.734587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:52:32.734587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:32.734587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 00:52:32.734587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:32.734587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 450/892] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Sep 14 00:52:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:52:32.735400 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd7ee9-bb40-43ac-bb41-f400b73d730b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:32.738184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:32.743190 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3cb24-3217-4c40-ac22-730ef81824cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:32.743839 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3cb24-3217-4c40-ac22-730ef81824cd None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:32.744119 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3cb24-3217-4c40-ac22-730ef81824cd None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:32.744397 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3cb24-3217-4c40-ac22-730ef81824cd None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:32.745027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3cb24-3217-4c40-ac22-730ef81824cd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:32.745331 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3cb24-3217-4c40-ac22-730ef81824cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:32.746706 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3cb24-3217-4c40-ac22-730ef81824cd None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:32.752128 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 443/893] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 00:52:32 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:32.752913 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3cb24-3217-4c40-ac22-730ef81824cd None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:32.832637 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:32.849644 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd7ee9-bb40-43ac-bb41-f400b73d730b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:32.870141 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3cb24-3217-4c40-ac22-730ef81824cd None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:32.888025 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d5f8c-e28f-425b-b0f9-daa12c1a7066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:32.888655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d5f8c-e28f-425b-b0f9-daa12c1a7066 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:32.888932 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d5f8c-e28f-425b-b0f9-daa12c1a7066 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:32.889202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d5f8c-e28f-425b-b0f9-daa12c1a7066 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:32.900893 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d5f8c-e28f-425b-b0f9-daa12c1a7066 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:52:32.906311 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada923ca-de80-4158-9b85-95a7b94508f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:32.907003 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada923ca-de80-4158-9b85-95a7b94508f2 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:32.907281 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada923ca-de80-4158-9b85-95a7b94508f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:32.907573 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada923ca-de80-4158-9b85-95a7b94508f2 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:32.908122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-b96d5f8c-e28f-425b-b0f9-daa12c1a7066 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 00:52:32.908122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:32.908122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:32.908122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:32.908122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:32.908122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:32.908122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:32.908122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:32.908122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:32.908122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:32.908122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:32.908122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:32.908122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:52:32.908122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:52:32.908122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:52:32.908122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:52:32.909060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:52:32.909060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:52:32.909060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:52:32.909060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:52:32.909060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:32.909060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:32.909060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:32.909060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:32.909060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:32.909060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:32.909060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:32.909060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:32.909060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:32.909060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:32.909060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:32.909060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:32.912618 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:32.912618 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:32.912618 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:32.912618 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:32.912618 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:52:32.912618 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:52:32.912618 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:52:32.912618 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:52:32.912618 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:52:32.912618 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:52:32.912618 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 00:52:32.912618 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:32.913287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada923ca-de80-4158-9b85-95a7b94508f2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:32.913287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada923ca-de80-4158-9b85-95a7b94508f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:32.913420 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 451/894] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Sep 14 00:52:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:32.913489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada923ca-de80-4158-9b85-95a7b94508f2 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:32.913564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d5f8c-e28f-425b-b0f9-daa12c1a7066 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:32.917293 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 444/895] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Sep 14 00:52:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:32.918085 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada923ca-de80-4158-9b85-95a7b94508f2 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:32.985904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:33.031183 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d5f8c-e28f-425b-b0f9-daa12c1a7066 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:33.036716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada923ca-de80-4158-9b85-95a7b94508f2 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:33.067097 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:33.067456 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77af291-0f15-448b-985e-0be9f6b4c5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:33.068100 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77af291-0f15-448b-985e-0be9f6b4c5f4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:33.068382 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77af291-0f15-448b-985e-0be9f6b4c5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:33.068656 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77af291-0f15-448b-985e-0be9f6b4c5f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:33.069368 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77af291-0f15-448b-985e-0be9f6b4c5f4 None admin] RBAC: Adding query filter params (interface=public, service_id=1e17d4025a3844249e3652d52b8e3fbf, region_id=RegionOne) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:33.069773 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77af291-0f15-448b-985e-0be9f6b4c5f4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:33.071250 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77af291-0f15-448b-985e-0be9f6b4c5f4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:33.073153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b470b-e3df-4e11-95e4-32409d0bc1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:33.073863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b470b-e3df-4e11-95e4-32409d0bc1f3 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:33.074072 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b470b-e3df-4e11-95e4-32409d0bc1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:33.074314 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b470b-e3df-4e11-95e4-32409d0bc1f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:33.074977 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b470b-e3df-4e11-95e4-32409d0bc1f3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:33.075278 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b470b-e3df-4e11-95e4-32409d0bc1f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:33.076673 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b470b-e3df-4e11-95e4-32409d0bc1f3 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:33.082388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 452/896] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Sep 14 00:52:32 2022] GET /identity/v3/endpoints?service_id=1e17d4025a3844249e3652d52b8e3fbf&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:33.082388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77af291-0f15-448b-985e-0be9f6b4c5f4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:33.091239 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 445/897] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Sep 14 00:52:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:33.096069 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b470b-e3df-4e11-95e4-32409d0bc1f3 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:33.190268 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77af291-0f15-448b-985e-0be9f6b4c5f4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:33.198997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b470b-e3df-4e11-95e4-32409d0bc1f3 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:33.215732 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:33.226531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6fb94-dd19-4765-8fb3-bf6ada21a7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:33.227144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6fb94-dd19-4765-8fb3-bf6ada21a7a2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:33.227426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6fb94-dd19-4765-8fb3-bf6ada21a7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:33.227735 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6fb94-dd19-4765-8fb3-bf6ada21a7a2 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:33.228365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6fb94-dd19-4765-8fb3-bf6ada21a7a2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:33.228668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6fb94-dd19-4765-8fb3-bf6ada21a7a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:33.230035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6fb94-dd19-4765-8fb3-bf6ada21a7a2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:33.234610 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 453/898] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Sep 14 00:52:33 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:33.235362 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6fb94-dd19-4765-8fb3-bf6ada21a7a2 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:33.245092 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364747b-bdcc-4e5a-ad4f-1ce5a8da29dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:33.246063 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364747b-bdcc-4e5a-ad4f-1ce5a8da29dc None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:33.246593 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364747b-bdcc-4e5a-ad4f-1ce5a8da29dc None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:33.246940 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364747b-bdcc-4e5a-ad4f-1ce5a8da29dc None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:33.247585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364747b-bdcc-4e5a-ad4f-1ce5a8da29dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:33.249530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364747b-bdcc-4e5a-ad4f-1ce5a8da29dc None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:33.258365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 446/899] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Sep 14 00:52:33 2022] GET /identity/v3/services => generated 1268 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 00:52:33.379667 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6fb94-dd19-4765-8fb3-bf6ada21a7a2 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:33.422751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3d084-845d-42bc-87a7-a4e74d5573ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:33.423415 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3d084-845d-42bc-87a7-a4e74d5573ea None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:33.423737 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3d084-845d-42bc-87a7-a4e74d5573ea None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:33.424017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3d084-845d-42bc-87a7-a4e74d5573ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:33.438306 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e3d084-845d-42bc-87a7-a4e74d5573ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:33.439900 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e3d084-845d-42bc-87a7-a4e74d5573ea None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:33.456096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-83e3d084-845d-42bc-87a7-a4e74d5573ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:33.456096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:33.456096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:33.456096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:33.456096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:33.456096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:33.456096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:33.456096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:33.456096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:33.456096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:33.456096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:33.456096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:33.456096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:33.456096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:33.456096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:33.456096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:33.457092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:33.457092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:33.457092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:33.457092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:33.457092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:33.457092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:33.457092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:33.457092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:33.457092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:33.457092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:33.457092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:33.457092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:33.457092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:33.457092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:33.457092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:33.457092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:33.457092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:33.458102 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:33.458102 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:33.458102 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:33.458102 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 454/900] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:33.463019 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364747b-bdcc-4e5a-ad4f-1ce5a8da29dc None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:33.528053 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:33.572337 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364747b-bdcc-4e5a-ad4f-1ce5a8da29dc None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:33.606253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a232fe-cd14-41d7-a678-cf0a944753dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:33.606862 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a232fe-cd14-41d7-a678-cf0a944753dd None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:33.607137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a232fe-cd14-41d7-a678-cf0a944753dd None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:33.607406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a232fe-cd14-41d7-a678-cf0a944753dd None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:33.608075 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a232fe-cd14-41d7-a678-cf0a944753dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:33.608909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a232fe-cd14-41d7-a678-cf0a944753dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:33.610417 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a232fe-cd14-41d7-a678-cf0a944753dd None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:33.622560 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 447/901] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:33.627891 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3d084-845d-42bc-87a7-a4e74d5573ea None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:33.676339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:33.738880 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3d084-845d-42bc-87a7-a4e74d5573ea None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:33.775199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f70f7-08bc-44b3-9cc6-ffac2047a469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:33.775884 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f70f7-08bc-44b3-9cc6-ffac2047a469 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:33.776164 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f70f7-08bc-44b3-9cc6-ffac2047a469 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:33.776448 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f70f7-08bc-44b3-9cc6-ffac2047a469 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:33.784640 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f70f7-08bc-44b3-9cc6-ffac2047a469 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:33.786033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f70f7-08bc-44b3-9cc6-ffac2047a469 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:33.799012 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-574f70f7-08bc-44b3-9cc6-ffac2047a469 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 00:52:33.799012 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:33.799012 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:33.799012 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:33.799012 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:33.799012 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:33.799012 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:33.799012 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:33.799012 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:33.799012 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:33.799012 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:33.799012 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:33.799012 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:52:33.799012 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:52:33.799012 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:52:33.799012 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:52:33.800049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:33.800049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:33.800049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:52:33.800049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:33.800049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:52:33.800049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:33.800049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:33.800049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:33.800049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:33.800049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:33.800049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:33.800049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:33.800049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:33.800049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:33.800049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:33.800049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:33.800049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:33.800049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:33.801014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:33.801014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:33.801014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:52:33.801014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:52:33.801014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:52:33.801014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:52:33.801014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:52:33.801014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:52:33.801014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 00:52:33.801014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:33.801014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 455/902] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Sep 14 00:52:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:33.806262 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a232fe-cd14-41d7-a678-cf0a944753dd None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:33.914453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a232fe-cd14-41d7-a678-cf0a944753dd None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:33.933589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:33.953974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdf55d-4fb8-4138-8b8b-03edcaccf4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:33.954925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdf55d-4fb8-4138-8b8b-03edcaccf4aa None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:33.955392 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdf55d-4fb8-4138-8b8b-03edcaccf4aa None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:33.955872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efdf55d-4fb8-4138-8b8b-03edcaccf4aa None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:33.956800 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efdf55d-4fb8-4138-8b8b-03edcaccf4aa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:33.957291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efdf55d-4fb8-4138-8b8b-03edcaccf4aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:33.958924 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efdf55d-4fb8-4138-8b8b-03edcaccf4aa None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:33.978836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 448/903] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Sep 14 00:52:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:33.985124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f70f7-08bc-44b3-9cc6-ffac2047a469 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:34.097552 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f70f7-08bc-44b3-9cc6-ffac2047a469 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:34.099906 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:34.136287 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f918a37-6a37-4b99-a3f3-6a076ae1b36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:34.137213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f918a37-6a37-4b99-a3f3-6a076ae1b36d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:34.137706 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f918a37-6a37-4b99-a3f3-6a076ae1b36d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:34.138163 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f918a37-6a37-4b99-a3f3-6a076ae1b36d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:34.212777 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f918a37-6a37-4b99-a3f3-6a076ae1b36d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:34.214587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f918a37-6a37-4b99-a3f3-6a076ae1b36d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:34.331778 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 456/904] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:52:33 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 00:52:34.333781 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdf55d-4fb8-4138-8b8b-03edcaccf4aa None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:34.418902 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4626d97-c946-40ab-8edb-3d8f5f688d79 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:34.419495 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4626d97-c946-40ab-8edb-3d8f5f688d79 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:34.420030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4626d97-c946-40ab-8edb-3d8f5f688d79 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:34.422839 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 457/905] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:34.427720 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaea959-9323-4d0c-b9d9-6229158b24fc None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:34.428010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaea959-9323-4d0c-b9d9-6229158b24fc None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:34.428285 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaea959-9323-4d0c-b9d9-6229158b24fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:34.434876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:34.447354 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efdf55d-4fb8-4138-8b8b-03edcaccf4aa None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:34.447975 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-daaea959-9323-4d0c-b9d9-6229158b24fc None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:34.449172 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-daaea959-9323-4d0c-b9d9-6229158b24fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:34.492756 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e76cb4-a2ab-4ca7-82da-f94f095d6e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:34.493399 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e76cb4-a2ab-4ca7-82da-f94f095d6e07 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:34.493683 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e76cb4-a2ab-4ca7-82da-f94f095d6e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:34.493962 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e76cb4-a2ab-4ca7-82da-f94f095d6e07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:34.504599 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e76cb4-a2ab-4ca7-82da-f94f095d6e07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:34.506109 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e76cb4-a2ab-4ca7-82da-f94f095d6e07 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:34.523917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-69e76cb4-a2ab-4ca7-82da-f94f095d6e07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:34.523917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:34.523917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:34.523917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:34.523917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:34.523917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:34.523917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:34.523917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:34.523917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:34.523917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:34.523917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:34.523917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:34.523917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:34.523917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:34.523917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:34.523917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:34.525153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:34.525153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:34.525153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:34.525153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:34.525153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:34.525153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:34.525153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:34.525153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:34.525153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:34.525153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:34.525153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:34.525153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:34.525153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:34.525153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:34.525153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:34.525153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:34.525153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:34.526125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:34.526125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:34.526125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:34.529807 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 449/906] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:34.530905 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e76cb4-a2ab-4ca7-82da-f94f095d6e07 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:34.536950 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 458/907] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 00:52:34.565456 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:34.645200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e76cb4-a2ab-4ca7-82da-f94f095d6e07 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:34.680681 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d9928-b5e2-49a4-bbf9-276e53e68172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:34.681570 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d9928-b5e2-49a4-bbf9-276e53e68172 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:34.682024 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d9928-b5e2-49a4-bbf9-276e53e68172 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:34.682463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08d9928-b5e2-49a4-bbf9-276e53e68172 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:34.683262 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d9928-b5e2-49a4-bbf9-276e53e68172 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:34.683758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d9928-b5e2-49a4-bbf9-276e53e68172 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:34.685280 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08d9928-b5e2-49a4-bbf9-276e53e68172 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:34.698621 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 450/908] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:34.704679 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaea959-9323-4d0c-b9d9-6229158b24fc None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:34.808232 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaea959-9323-4d0c-b9d9-6229158b24fc None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:34.824572 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d9928-b5e2-49a4-bbf9-276e53e68172 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:34.843991 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:34.845512 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2b0c7-b35e-4b40-a860-224906888a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:34.846350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2b0c7-b35e-4b40-a860-224906888a47 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:34.846544 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2b0c7-b35e-4b40-a860-224906888a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:34.846837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2b0c7-b35e-4b40-a860-224906888a47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:34.854016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2b0c7-b35e-4b40-a860-224906888a47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:34.855647 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2b0c7-b35e-4b40-a860-224906888a47 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:34.867484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-bfd2b0c7-b35e-4b40-a860-224906888a47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:34.867484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:34.867484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:34.867484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:34.867484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:34.867484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:34.867484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:34.867484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:34.867484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:34.867484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:34.867484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:34.867484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:34.867484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:52:34.867484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:52:34.867484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:52:34.868422 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:52:34.868422 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:34.868422 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:34.868422 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:34.868422 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:34.868422 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:34.868422 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:34.868422 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:34.868422 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:34.868422 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:34.868422 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:34.868422 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:34.868422 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:34.868422 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:34.868422 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:34.868422 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:34.868422 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:34.869437 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:52:34.869437 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:52:34.869437 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:52:34.869437 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:52:34.869437 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:52:34.869437 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:52:34.869437 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:34.869437 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:34.873632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 459/909] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:34.875141 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2b0c7-b35e-4b40-a860-224906888a47 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:34.941693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:34.959696 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08d9928-b5e2-49a4-bbf9-276e53e68172 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:34.996258 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e4fc2c-01ba-475d-b43b-be5fab55ab27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:34.997286 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e4fc2c-01ba-475d-b43b-be5fab55ab27 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:34.997790 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e4fc2c-01ba-475d-b43b-be5fab55ab27 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:34.998268 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e4fc2c-01ba-475d-b43b-be5fab55ab27 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:34.999635 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e4fc2c-01ba-475d-b43b-be5fab55ab27 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:35.001436 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e4fc2c-01ba-475d-b43b-be5fab55ab27 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:35.008174 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2b0c7-b35e-4b40-a860-224906888a47 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:35.010965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-24e4fc2c-01ba-475d-b43b-be5fab55ab27 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 00:52:35.010965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:35.010965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:52:35.010965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:52:35.010965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:52:35.010965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:52:35.010965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:52:35.010965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:35.010965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 00:52:35.010965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:35.010965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:35.010965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:35.010965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:35.010965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:35.010965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:35.010965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:35.011915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:35.011915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:35.011915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:35.011915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:35.011915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:35.011915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:35.011915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:35.011915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:52:35.011915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:52:35.011915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:52:35.011915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:52:35.011915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:35.011915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:35.011915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:35.011915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:35.011915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:35.011915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:35.012844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:35.012844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:35.012844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:35.012844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:35.012844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:35.012844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:35.012844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:35.012844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:35.012844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:35.012844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:35.012844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:52:35.012844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:35.012844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 00:52:35.012844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:35.015946 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 451/910] 10.222.0.42 () {56 vars in 1132 bytes} [Wed Sep 14 00:52:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:35.021305 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e4fc2c-01ba-475d-b43b-be5fab55ab27 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:35.047334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a37e6-b795-421a-9353-af3ab1bfad38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:35.048014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a37e6-b795-421a-9353-af3ab1bfad38 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:35.048299 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a37e6-b795-421a-9353-af3ab1bfad38 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:35.048576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a37e6-b795-421a-9353-af3ab1bfad38 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:35.049275 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a37e6-b795-421a-9353-af3ab1bfad38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:35.049582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a37e6-b795-421a-9353-af3ab1bfad38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:35.051013 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a37e6-b795-421a-9353-af3ab1bfad38 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:35.064490 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 460/911] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:52:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:35.069999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a37e6-b795-421a-9353-af3ab1bfad38 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:35.137212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:35.146689 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e4fc2c-01ba-475d-b43b-be5fab55ab27 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:35.182417 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0935d8-f685-4cc1-819b-86a90e3ed976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:35.183178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0935d8-f685-4cc1-819b-86a90e3ed976 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:35.183580 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0935d8-f685-4cc1-819b-86a90e3ed976 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:35.184006 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0935d8-f685-4cc1-819b-86a90e3ed976 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:35.184742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0935d8-f685-4cc1-819b-86a90e3ed976 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:35.185149 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0935d8-f685-4cc1-819b-86a90e3ed976 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:35.187226 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0935d8-f685-4cc1-819b-86a90e3ed976 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:35.192642 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 452/912] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Sep 14 00:52:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:35.197742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0935d8-f685-4cc1-819b-86a90e3ed976 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:35.206461 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a37e6-b795-421a-9353-af3ab1bfad38 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:35.220834 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:35.248346 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0eaa0-fd65-4c93-8b28-79075b4fb7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:35.250147 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0eaa0-fd65-4c93-8b28-79075b4fb7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:35.250942 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0eaa0-fd65-4c93-8b28-79075b4fb7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:35.251445 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0eaa0-fd65-4c93-8b28-79075b4fb7e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:35.252413 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc0eaa0-fd65-4c93-8b28-79075b4fb7e0 None admin] RBAC: Adding query filter params (role.id=ae888494799042399af96f094898c4ed, scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3, user.id=ceccc909529042539dd2d5015058d8f7) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:35.252947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc0eaa0-fd65-4c93-8b28-79075b4fb7e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:35.255365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc0eaa0-fd65-4c93-8b28-79075b4fb7e0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:35.268777 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 461/913] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:52:35 2022] GET /identity/v3/role_assignments?user.id=ceccc909529042539dd2d5015058d8f7&scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3&role.id=ae888494799042399af96f094898c4ed => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:35.310040 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0935d8-f685-4cc1-819b-86a90e3ed976 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:35.349474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965cb889-91d0-4572-8b31-bedb5050996d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:35.350138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965cb889-91d0-4572-8b31-bedb5050996d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:35.350366 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965cb889-91d0-4572-8b31-bedb5050996d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:35.350640 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965cb889-91d0-4572-8b31-bedb5050996d None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:35.351282 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965cb889-91d0-4572-8b31-bedb5050996d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:35.351610 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965cb889-91d0-4572-8b31-bedb5050996d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:35.353009 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965cb889-91d0-4572-8b31-bedb5050996d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:35.359805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 453/914] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Sep 14 00:52:35 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:35.368727 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0eaa0-fd65-4c93-8b28-79075b4fb7e0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:35.490254 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0eaa0-fd65-4c93-8b28-79075b4fb7e0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:35.524107 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:35.528168 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c575e-f7f6-4997-8ee8-44d08791ea07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:35.529056 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c575e-f7f6-4997-8ee8-44d08791ea07 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:35.529589 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c575e-f7f6-4997-8ee8-44d08791ea07 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:35.530071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c575e-f7f6-4997-8ee8-44d08791ea07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:35.531787 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c575e-f7f6-4997-8ee8-44d08791ea07 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:35.533458 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c575e-f7f6-4997-8ee8-44d08791ea07 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:35.541786 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc83a96-ffe0-4af5-9ff6-33d612ebcd8b None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:35.542119 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc83a96-ffe0-4af5-9ff6-33d612ebcd8b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:35.542354 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc83a96-ffe0-4af5-9ff6-33d612ebcd8b None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:35.543997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 454/915] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:52:35.549933 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79587a18-5c5b-4ee0-9902-adddcda3103b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:35.550489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79587a18-5c5b-4ee0-9902-adddcda3103b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:35.551073 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79587a18-5c5b-4ee0-9902-adddcda3103b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:35.565245 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 462/916] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 00:52:35 2022] POST /identity/v3/endpoints => generated 347 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:52:35.578929 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-79587a18-5c5b-4ee0-9902-adddcda3103b None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:35.580684 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-79587a18-5c5b-4ee0-9902-adddcda3103b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:35.587193 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c91f7f-5ee3-49b4-967e-8e19913e447a None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:35.587534 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c91f7f-5ee3-49b4-967e-8e19913e447a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:35.587778 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c91f7f-5ee3-49b4-967e-8e19913e447a None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:35.590744 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 463/917] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:52:35.595473 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09dcaf-4801-4b09-8fd9-d84ba2a07fe3 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:35.595813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09dcaf-4801-4b09-8fd9-d84ba2a07fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:35.596075 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09dcaf-4801-4b09-8fd9-d84ba2a07fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:35.616066 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-8a09dcaf-4801-4b09-8fd9-d84ba2a07fe3 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:35.617134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8a09dcaf-4801-4b09-8fd9-d84ba2a07fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:35.672201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 455/918] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:35 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 00:52:35.703812 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 464/919] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:35 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 00:52:35.942554 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79587a18-5c5b-4ee0-9902-adddcda3103b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:35.950905 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:35.980219 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09dcaf-4801-4b09-8fd9-d84ba2a07fe3 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:36.062275 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79587a18-5c5b-4ee0-9902-adddcda3103b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:36.101778 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09dcaf-4801-4b09-8fd9-d84ba2a07fe3 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:36.103574 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:36.109204 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2046186-f797-4e3d-a367-02be34e71dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:36.109858 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2046186-f797-4e3d-a367-02be34e71dff None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:36.110204 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2046186-f797-4e3d-a367-02be34e71dff None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:36.110488 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2046186-f797-4e3d-a367-02be34e71dff None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:36.111076 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2046186-f797-4e3d-a367-02be34e71dff None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:36.112617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2046186-f797-4e3d-a367-02be34e71dff None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:36.124487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-d2046186-f797-4e3d-a367-02be34e71dff None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 00:52:36.124487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:36.124487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:52:36.124487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:52:36.124487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:52:36.124487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:52:36.124487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:52:36.124487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:36.124487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 00:52:36.124487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:36.124487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:36.124487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:36.124487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:36.124487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:36.124487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:36.124487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:36.124487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:36.125533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:36.125533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:36.125533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:36.125533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:36.125533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:36.125533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:36.125533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:52:36.125533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:52:36.125533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:52:36.125533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:52:36.125533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:36.125533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:36.125533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:36.125533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:36.125533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:36.125533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:36.126400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:36.126400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:36.126400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:36.126400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:36.126400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:36.126400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:36.126400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:36.126400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:36.126400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:36.126400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:36.126400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:52:36.126400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:36.126400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 00:52:36.126400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:36.126400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 456/920] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 00:52:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:36.133428 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2046186-f797-4e3d-a367-02be34e71dff None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:36.148897 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb9848-5afb-424c-87e3-58abd3a9d877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:36.150665 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb9848-5afb-424c-87e3-58abd3a9d877 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:36.151155 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb9848-5afb-424c-87e3-58abd3a9d877 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:36.151888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb9848-5afb-424c-87e3-58abd3a9d877 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:36.152568 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb9848-5afb-424c-87e3-58abd3a9d877 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:36.153953 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb9848-5afb-424c-87e3-58abd3a9d877 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:36.169637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 465/921] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 00:52:35 2022] POST /identity/v3/services => generated 241 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:52:36.230846 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:36.272251 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2046186-f797-4e3d-a367-02be34e71dff None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:36.318691 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b705f-969b-4c69-8df5-7a58cc0615fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:36.319653 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b705f-969b-4c69-8df5-7a58cc0615fe None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:36.320177 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b705f-969b-4c69-8df5-7a58cc0615fe None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:36.320603 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b705f-969b-4c69-8df5-7a58cc0615fe None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:36.321565 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b705f-969b-4c69-8df5-7a58cc0615fe None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:36.322206 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b705f-969b-4c69-8df5-7a58cc0615fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:36.323893 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b705f-969b-4c69-8df5-7a58cc0615fe None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:36.330274 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 457/922] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 00:52:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:36.336414 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb9848-5afb-424c-87e3-58abd3a9d877 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:36.448365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb9848-5afb-424c-87e3-58abd3a9d877 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:36.494988 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c46037b-c6df-4672-9c70-210e4096ab84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:36.495539 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46037b-c6df-4672-9c70-210e4096ab84 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:36.495843 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46037b-c6df-4672-9c70-210e4096ab84 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:36.496121 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c46037b-c6df-4672-9c70-210e4096ab84 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:36.496753 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c46037b-c6df-4672-9c70-210e4096ab84 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:36.497054 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c46037b-c6df-4672-9c70-210e4096ab84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:36.498425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c46037b-c6df-4672-9c70-210e4096ab84 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:36.504806 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 466/923] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 00:52:36 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:36.510223 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b705f-969b-4c69-8df5-7a58cc0615fe None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:36.621683 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b705f-969b-4c69-8df5-7a58cc0615fe None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:36.656056 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:36.660599 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc22dcb4-7bb8-4f0e-980d-688769a392f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:36.661456 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22dcb4-7bb8-4f0e-980d-688769a392f5 None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:36.662007 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22dcb4-7bb8-4f0e-980d-688769a392f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:36.662488 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc22dcb4-7bb8-4f0e-980d-688769a392f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:36.663639 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc22dcb4-7bb8-4f0e-980d-688769a392f5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:36.665314 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc22dcb4-7bb8-4f0e-980d-688769a392f5 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:36.691664 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 458/924] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 00:52:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:52:37.544189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386ddc5-6825-4177-8734-eca154d35294 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:37.544736 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386ddc5-6825-4177-8734-eca154d35294 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:37.544825 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386ddc5-6825-4177-8734-eca154d35294 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:37.547030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 467/925] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:37.552396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b0bdf-805f-416a-b14a-d982ebf9a405 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:37.552820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b0bdf-805f-416a-b14a-d982ebf9a405 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:37.553011 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b0bdf-805f-416a-b14a-d982ebf9a405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:37.572037 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-914b0bdf-805f-416a-b14a-d982ebf9a405 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:37.573234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-914b0bdf-805f-416a-b14a-d982ebf9a405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:37.653791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 459/926] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:37.822875 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087904a-90ab-412e-a88b-2b4a76196de0 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:37.823065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087904a-90ab-412e-a88b-2b4a76196de0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:37.823356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087904a-90ab-412e-a88b-2b4a76196de0 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:37.824977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 468/927] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:37.830242 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5909a-536a-4e75-bac6-a469043c97cd None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:37.830533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5909a-536a-4e75-bac6-a469043c97cd None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:37.830809 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5909a-536a-4e75-bac6-a469043c97cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:37.849706 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d9a5909a-536a-4e75-bac6-a469043c97cd None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:37.850889 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d9a5909a-536a-4e75-bac6-a469043c97cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:37.920861 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 460/928] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:37.928464 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087904a-90ab-412e-a88b-2b4a76196de0 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:37.957371 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:38.039005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087904a-90ab-412e-a88b-2b4a76196de0 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:38.081577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962530c-f013-45a0-b393-77b0ac36529b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:38.082292 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962530c-f013-45a0-b393-77b0ac36529b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:38.082575 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962530c-f013-45a0-b393-77b0ac36529b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:38.082848 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962530c-f013-45a0-b393-77b0ac36529b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:38.094995 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0962530c-f013-45a0-b393-77b0ac36529b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:52:38.099859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-0962530c-f013-45a0-b393-77b0ac36529b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 00:52:38.099859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:38.099859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:38.099859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:38.099859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:38.099859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:38.099859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:38.099859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:38.099859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:38.099859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:38.099859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:38.099859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:38.099859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:52:38.099859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:52:38.099859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:52:38.099859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:52:38.100790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:52:38.100790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:52:38.100790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:52:38.100790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:52:38.100790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:38.100790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:38.100790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:38.100790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:38.100790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:38.100790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:38.100790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:38.100790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:38.100790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:38.100790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:38.100790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:38.100790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:38.101638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:38.101638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:38.101638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:38.101638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:38.101638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:52:38.101638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:52:38.101638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:52:38.101638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:52:38.101638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:52:38.101638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:52:38.101638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 00:52:38.101638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:38.101638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 469/929] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Sep 14 00:52:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:38.113807 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5909a-536a-4e75-bac6-a469043c97cd None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:38.160311 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:38.198130 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962530c-f013-45a0-b393-77b0ac36529b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:38.225210 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5909a-536a-4e75-bac6-a469043c97cd None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:38.261949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e4c75-0f7d-4a2d-9e26-bb6ffdce3393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:38.262437 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e4c75-0f7d-4a2d-9e26-bb6ffdce3393 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:38.262717 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e4c75-0f7d-4a2d-9e26-bb6ffdce3393 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:38.262983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e4c75-0f7d-4a2d-9e26-bb6ffdce3393 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:38.263701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e4c75-0f7d-4a2d-9e26-bb6ffdce3393 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:38.263963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e4c75-0f7d-4a2d-9e26-bb6ffdce3393 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:38.266338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e4c75-0f7d-4a2d-9e26-bb6ffdce3393 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:38.275628 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 461/930] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Sep 14 00:52:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:38.280779 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e4c75-0f7d-4a2d-9e26-bb6ffdce3393 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:38.301571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:38.322911 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962530c-f013-45a0-b393-77b0ac36529b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:38.368039 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c70e06-5d6d-4a47-98b7-aefbf27e9b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:38.369185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c70e06-5d6d-4a47-98b7-aefbf27e9b88 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:38.369647 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c70e06-5d6d-4a47-98b7-aefbf27e9b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:38.370079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c70e06-5d6d-4a47-98b7-aefbf27e9b88 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:38.371059 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c70e06-5d6d-4a47-98b7-aefbf27e9b88 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:38.373059 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c70e06-5d6d-4a47-98b7-aefbf27e9b88 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:38.387549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-98c70e06-5d6d-4a47-98b7-aefbf27e9b88 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 00:52:38.387549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:38.387549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:52:38.387549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:52:38.387549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:52:38.387549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:52:38.387549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:52:38.387549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:38.387549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 00:52:38.387549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:38.387549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:38.387549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:38.387549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:38.387549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:38.387549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:38.387549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:38.387549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:38.388841 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:38.388841 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:38.388841 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:38.388841 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:38.388841 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:38.388841 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:38.388841 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:52:38.388841 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:52:38.388841 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:52:38.388841 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:52:38.388841 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:38.388841 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:38.388841 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:38.388841 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:38.388841 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:38.388841 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:38.390655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:38.390655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:38.390655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:38.390655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:38.390655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:38.390655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:38.390655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:38.390655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:38.390655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:38.390655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:38.390655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:52:38.390655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:38.390655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 00:52:38.390655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:38.394350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 470/931] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:52:38.400262 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c70e06-5d6d-4a47-98b7-aefbf27e9b88 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:38.405528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e4c75-0f7d-4a2d-9e26-bb6ffdce3393 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:38.409199 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:38.444850 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8cd4eb-28c2-4e8a-9ca3-56a489e5fdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:38.445675 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8cd4eb-28c2-4e8a-9ca3-56a489e5fdfe None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:38.446027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8cd4eb-28c2-4e8a-9ca3-56a489e5fdfe None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:38.446372 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8cd4eb-28c2-4e8a-9ca3-56a489e5fdfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:38.458874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8cd4eb-28c2-4e8a-9ca3-56a489e5fdfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:38.460588 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8cd4eb-28c2-4e8a-9ca3-56a489e5fdfe None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:38.483670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-ff8cd4eb-28c2-4e8a-9ca3-56a489e5fdfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:38.483670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:38.483670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:38.483670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:38.483670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:38.483670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:38.483670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:38.483670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:38.483670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:38.483670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:38.483670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:38.483670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:38.483670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:38.483670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:38.483670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:38.483670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:38.484887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:38.484887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:38.484887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:38.484887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:38.484887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:38.484887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:38.484887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:38.484887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:38.484887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:38.484887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:38.484887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:38.484887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:38.484887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:38.484887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:38.484887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:38.484887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:38.484887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:38.485808 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:38.485808 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:38.485808 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:38.489798 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 462/932] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:38.490767 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8cd4eb-28c2-4e8a-9ca3-56a489e5fdfe None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:38.529910 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c70e06-5d6d-4a47-98b7-aefbf27e9b88 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:38.565729 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca04d2-881a-4188-8f03-5e8c8efca3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:38.566379 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca04d2-881a-4188-8f03-5e8c8efca3aa None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:38.566602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca04d2-881a-4188-8f03-5e8c8efca3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:38.566876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca04d2-881a-4188-8f03-5e8c8efca3aa None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:38.567502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca04d2-881a-4188-8f03-5e8c8efca3aa None admin] RBAC: Adding query filter params (name=compute) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:38.567831 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca04d2-881a-4188-8f03-5e8c8efca3aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:38.569302 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca04d2-881a-4188-8f03-5e8c8efca3aa None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:38.574025 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 471/933] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 00:52:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:38.575165 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7fc4e-2489-4e47-8d41-a3ac7b3bd079 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:38.575440 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7fc4e-2489-4e47-8d41-a3ac7b3bd079 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:38.575789 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7fc4e-2489-4e47-8d41-a3ac7b3bd079 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:38.577263 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 472/934] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:38.582674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7fc4e-2489-4e47-8d41-a3ac7b3bd079 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:38.598807 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:38.610127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8cd4eb-28c2-4e8a-9ca3-56a489e5fdfe None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:38.647729 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9714ca-2a06-40da-9ee1-4c58b78fd065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:38.648729 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9714ca-2a06-40da-9ee1-4c58b78fd065 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:38.649278 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9714ca-2a06-40da-9ee1-4c58b78fd065 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:38.649712 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9714ca-2a06-40da-9ee1-4c58b78fd065 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:38.650513 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9714ca-2a06-40da-9ee1-4c58b78fd065 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:38.650963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9714ca-2a06-40da-9ee1-4c58b78fd065 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:38.652663 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9714ca-2a06-40da-9ee1-4c58b78fd065 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:38.670341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 463/935] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:38.670341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a487ad-cabc-460f-8060-0474524ec016 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:38.670341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a487ad-cabc-460f-8060-0474524ec016 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:38.670341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a487ad-cabc-460f-8060-0474524ec016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:38.689177 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-85a487ad-cabc-460f-8060-0474524ec016 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:38.690595 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-85a487ad-cabc-460f-8060-0474524ec016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:38.699586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7fc4e-2489-4e47-8d41-a3ac7b3bd079 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:38.701471 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:38.738436 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f026f91-22cf-4b9b-99d8-16beab3b0243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:38.739052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f026f91-22cf-4b9b-99d8-16beab3b0243 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:38.739329 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f026f91-22cf-4b9b-99d8-16beab3b0243 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:38.739622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f026f91-22cf-4b9b-99d8-16beab3b0243 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:38.740255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f026f91-22cf-4b9b-99d8-16beab3b0243 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:38.740562 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f026f91-22cf-4b9b-99d8-16beab3b0243 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:38.742113 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f026f91-22cf-4b9b-99d8-16beab3b0243 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:38.747729 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 473/936] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 00:52:38 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:38.748510 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f026f91-22cf-4b9b-99d8-16beab3b0243 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:38.764809 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 464/937] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:38.850612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:38.880915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f026f91-22cf-4b9b-99d8-16beab3b0243 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:38.925504 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a550a-67a2-4aca-8e69-71b5e25bf8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:38.926166 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a550a-67a2-4aca-8e69-71b5e25bf8a9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:38.926448 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a550a-67a2-4aca-8e69-71b5e25bf8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:38.928200 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a550a-67a2-4aca-8e69-71b5e25bf8a9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:38.935439 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a550a-67a2-4aca-8e69-71b5e25bf8a9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:38.937021 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a550a-67a2-4aca-8e69-71b5e25bf8a9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:38.949603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-670a550a-67a2-4aca-8e69-71b5e25bf8a9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 00:52:38.949603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:38.949603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:38.949603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:38.949603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:38.949603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:38.949603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:38.949603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:38.949603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:38.949603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:38.949603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:38.949603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:38.949603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:52:38.949603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:52:38.949603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:52:38.949603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:52:38.950564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:38.950564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:38.950564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:52:38.950564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:38.950564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:52:38.950564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:38.950564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:38.950564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:38.950564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:38.950564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:38.950564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:38.950564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:38.950564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:38.950564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:38.950564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:38.950564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:38.950564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:38.950564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:38.951524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:38.951524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:38.951524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:52:38.951524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:52:38.951524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:52:38.951524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:52:38.951524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:52:38.951524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:52:38.951524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 00:52:38.951524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:38.951524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 474/938] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Sep 14 00:52:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:38.957312 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a487ad-cabc-460f-8060-0474524ec016 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:39.028188 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c268570-6a9e-4649-acbf-acfdb149267b None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:39.028765 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c268570-6a9e-4649-acbf-acfdb149267b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:39.029255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c268570-6a9e-4649-acbf-acfdb149267b None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:39.031168 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 475/939] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:39.036792 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c8ca4-14cf-4382-a6d0-9a3656ce40ab None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:39.037202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c8ca4-14cf-4382-a6d0-9a3656ce40ab None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:39.037578 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c8ca4-14cf-4382-a6d0-9a3656ce40ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:39.057258 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-739c8ca4-14cf-4382-a6d0-9a3656ce40ab None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:39.058987 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-739c8ca4-14cf-4382-a6d0-9a3656ce40ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:39.061692 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a487ad-cabc-460f-8060-0474524ec016 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:39.103454 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a76366-fe40-4e35-b20d-a74481b58ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:39.104094 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a76366-fe40-4e35-b20d-a74481b58ebb None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:39.104372 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a76366-fe40-4e35-b20d-a74481b58ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:39.104643 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a76366-fe40-4e35-b20d-a74481b58ebb None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:39.105371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a76366-fe40-4e35-b20d-a74481b58ebb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:39.105676 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a76366-fe40-4e35-b20d-a74481b58ebb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:39.107099 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a76366-fe40-4e35-b20d-a74481b58ebb None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:39.132645 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 465/940] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Sep 14 00:52:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:39.132645 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a76366-fe40-4e35-b20d-a74481b58ebb None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:39.138302 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:39.143833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 476/941] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 00:52:39.144709 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c8ca4-14cf-4382-a6d0-9a3656ce40ab None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:39.224222 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:39.268641 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a76366-fe40-4e35-b20d-a74481b58ebb None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:39.276147 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c8ca4-14cf-4382-a6d0-9a3656ce40ab None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:39.312592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430556c-c763-4cd6-8e7e-f91f186e0630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:39.312956 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab0b2bb-af03-4531-af9d-2cdb471f10d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:39.313282 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430556c-c763-4cd6-8e7e-f91f186e0630 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:39.313491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0b2bb-af03-4531-af9d-2cdb471f10d1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:39.313591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430556c-c763-4cd6-8e7e-f91f186e0630 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:39.313770 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0b2bb-af03-4531-af9d-2cdb471f10d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:39.313869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430556c-c763-4cd6-8e7e-f91f186e0630 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:39.314046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0b2bb-af03-4531-af9d-2cdb471f10d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:39.314355 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430556c-c763-4cd6-8e7e-f91f186e0630 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:39.316242 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430556c-c763-4cd6-8e7e-f91f186e0630 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:39.325111 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab0b2bb-af03-4531-af9d-2cdb471f10d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:39.326560 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab0b2bb-af03-4531-af9d-2cdb471f10d1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:39.327686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-f430556c-c763-4cd6-8e7e-f91f186e0630 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 00:52:39.327686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:39.327686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:52:39.327686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:52:39.327686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:52:39.327686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:52:39.327686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:52:39.327686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:39.327686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 00:52:39.327686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:39.327686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:39.327686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:39.327686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:39.327686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:39.327686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:39.327686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:39.327686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:39.328836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:39.328836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:39.328836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:39.328836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:39.328836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:39.328836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:39.328836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:52:39.328836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:52:39.328836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:52:39.328836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:52:39.328836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:39.328836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:39.328836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:39.328836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:39.328836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:39.328836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:39.329686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:39.329686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:39.329686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:39.329686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:39.329686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:39.329686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:39.329686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:39.329686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:39.329686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:39.329686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:39.329686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:52:39.329686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:39.329686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 00:52:39.329686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:39.331707 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 466/942] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Sep 14 00:52:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:39.335320 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430556c-c763-4cd6-8e7e-f91f186e0630 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:39.348629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-4ab0b2bb-af03-4531-af9d-2cdb471f10d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:39.348629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:39.348629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:39.348629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:39.348629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:39.348629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:39.348629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:39.348629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:39.348629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:39.348629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:39.348629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:39.348629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:39.348629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:39.348629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:39.348629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:39.348629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:39.349622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:39.349622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:39.349622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:39.349622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:39.349622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:39.349622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:39.349622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:39.349622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:39.349622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:39.349622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:39.349622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:39.349622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:39.349622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:39.349622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:39.349622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:39.349622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:39.349622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:39.350536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:39.350536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:39.350536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:39.356999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 477/943] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:39.358122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab0b2bb-af03-4531-af9d-2cdb471f10d1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:39.408594 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:39.451065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430556c-c763-4cd6-8e7e-f91f186e0630 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:39.474427 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab0b2bb-af03-4531-af9d-2cdb471f10d1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:39.493567 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143f907-8e46-4391-ac9f-6d728fac9de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:39.494179 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143f907-8e46-4391-ac9f-6d728fac9de6 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:39.494454 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143f907-8e46-4391-ac9f-6d728fac9de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:39.494722 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143f907-8e46-4391-ac9f-6d728fac9de6 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:39.495343 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4143f907-8e46-4391-ac9f-6d728fac9de6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:39.495670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4143f907-8e46-4391-ac9f-6d728fac9de6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:39.497039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4143f907-8e46-4391-ac9f-6d728fac9de6 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:39.501091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:39.503005 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 467/944] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Sep 14 00:52:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:39.504107 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143f907-8e46-4391-ac9f-6d728fac9de6 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:39.521905 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cccead-bd10-43cc-bce9-2e4cbef607c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:39.522528 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cccead-bd10-43cc-bce9-2e4cbef607c4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:39.522815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cccead-bd10-43cc-bce9-2e4cbef607c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:39.523089 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cccead-bd10-43cc-bce9-2e4cbef607c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:39.523786 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cccead-bd10-43cc-bce9-2e4cbef607c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:39.524095 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cccead-bd10-43cc-bce9-2e4cbef607c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:39.525468 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cccead-bd10-43cc-bce9-2e4cbef607c4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:39.538965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 478/945] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:39.539824 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cccead-bd10-43cc-bce9-2e4cbef607c4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:39.614052 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143f907-8e46-4391-ac9f-6d728fac9de6 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:39.644982 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:39.651175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc397a8-2b0d-4005-92ae-1fe6c4a29c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:39.651758 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cccead-bd10-43cc-bce9-2e4cbef607c4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:39.652192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc397a8-2b0d-4005-92ae-1fe6c4a29c0c None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:39.652192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc397a8-2b0d-4005-92ae-1fe6c4a29c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:39.652354 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc397a8-2b0d-4005-92ae-1fe6c4a29c0c None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:39.652992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc397a8-2b0d-4005-92ae-1fe6c4a29c0c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:39.654451 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc397a8-2b0d-4005-92ae-1fe6c4a29c0c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:39.657843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: WARNING keystone.server.flask.common [None req-9dc397a8-2b0d-4005-92ae-1fe6c4a29c0c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 14 00:52:39.675959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.sql.core [None req-9dc397a8-2b0d-4005-92ae-1fe6c4a29c0c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 14 00:52:39.675959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 14 00:52:39.675959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [parameters: {'id': '52f6627efbb4429f90c575dd9ab4bfda', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 14 00:52:39.675959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81418) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 14 00:52:39.682716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-9dc397a8-2b0d-4005-92ae-1fe6c4a29c0c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 52f6627efbb4429f90c575dd9ab4bfda.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 52f6627efbb4429f90c575dd9ab4bfda. Sep 14 00:52:39.682716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:39.682716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 00:52:39.682716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 00:52:39.682716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 00:52:39.682716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 00:52:39.682716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 00:52:39.682716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 00:52:39.682716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 00:52:39.682716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application conn.query(q) Sep 14 00:52:39.682716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 00:52:39.682716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 00:52:39.682716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 00:52:39.683606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application result.read() Sep 14 00:52:39.683606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 00:52:39.683606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 00:52:39.683606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 00:52:39.683606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 00:52:39.683606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 00:52:39.683606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 00:52:39.683606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 00:52:39.683606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 00:52:39.683606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 14 00:52:39.683606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:39.683606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 14 00:52:39.683606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:39.683606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:39.683606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 14 00:52:39.683606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 14 00:52:39.683606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 14 00:52:39.683606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 14 00:52:39.684614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 00:52:39.684614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application next(self.gen) Sep 14 00:52:39.684614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 14 00:52:39.684614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application yield resource Sep 14 00:52:39.684614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 00:52:39.684614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application next(self.gen) Sep 14 00:52:39.684614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 14 00:52:39.684614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self.session.rollback() Sep 14 00:52:39.684614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 14 00:52:39.684614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self.force_reraise() Sep 14 00:52:39.684614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 14 00:52:39.684614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise self.value Sep 14 00:52:39.684614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 14 00:52:39.684614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 14 00:52:39.684614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 14 00:52:39.684614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application session.commit() Sep 14 00:52:39.684614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 14 00:52:39.685902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 14 00:52:39.685902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 14 00:52:39.685902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._prepare_impl() Sep 14 00:52:39.685902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 14 00:52:39.685902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self.session.flush() Sep 14 00:52:39.685902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 14 00:52:39.685902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._flush(objects) Sep 14 00:52:39.685902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 14 00:52:39.685902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 14 00:52:39.685902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 14 00:52:39.685902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application compat.raise_( Sep 14 00:52:39.685902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 00:52:39.685902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception Sep 14 00:52:39.685902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 14 00:52:39.685902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application flush_context.execute() Sep 14 00:52:39.685902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 14 00:52:39.685902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rec.execute(self) Sep 14 00:52:39.687039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 14 00:52:39.687039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 14 00:52:39.687039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 14 00:52:39.687039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 14 00:52:39.687039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 14 00:52:39.687039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 14 00:52:39.687039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 14 00:52:39.687039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 14 00:52:39.687039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 14 00:52:39.687039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 14 00:52:39.687039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 14 00:52:39.687039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 14 00:52:39.687039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 14 00:52:39.687039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 14 00:52:39.687039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 14 00:52:39.688447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 14 00:52:39.688447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 00:52:39.688447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception Sep 14 00:52:39.688447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 00:52:39.688447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 00:52:39.688447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 00:52:39.688447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 00:52:39.688447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 00:52:39.688447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 00:52:39.688447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 00:52:39.688447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application conn.query(q) Sep 14 00:52:39.688447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 00:52:39.688447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 00:52:39.688447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 00:52:39.688447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application result.read() Sep 14 00:52:39.688447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 00:52:39.688447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 00:52:39.690062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 00:52:39.690062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 00:52:39.690062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 00:52:39.690062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 00:52:39.690062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 00:52:39.690062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 00:52:39.690062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 14 00:52:39.690062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 14 00:52:39.690062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [parameters: {'id': '52f6627efbb4429f90c575dd9ab4bfda', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 14 00:52:39.690062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 14 00:52:39.690062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:39.690062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:39.690062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:39.690062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:39.690062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 14 00:52:39.691867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 14 00:52:39.691867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 14 00:52:39.691867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 14 00:52:39.691867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 14 00:52:39.691867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:39.691867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:39.691867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:39.691867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:39.691867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:39.691867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:39.691867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:39.691867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:39.691867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:39.691867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:39.691867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:39.691867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:39.691867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:39.694045 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:39.694045 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 14 00:52:39.694045 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 14 00:52:39.694045 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:39.694045 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:39.694045 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 14 00:52:39.694045 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 14 00:52:39.694045 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 52f6627efbb4429f90c575dd9ab4bfda. Sep 14 00:52:39.694045 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:39.694045 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 468/946] 10.222.0.42 () {60 vars in 1140 bytes} [Wed Sep 14 00:52:39 2022] POST /identity/v3/projects => generated 262 bytes in 180 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:39.694045 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc397a8-2b0d-4005-92ae-1fe6c4a29c0c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:39.700302 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04afd8-a163-4859-9250-7a8cf0c4b27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:39.700971 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04afd8-a163-4859-9250-7a8cf0c4b27e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:39.701252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04afd8-a163-4859-9250-7a8cf0c4b27e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:39.701527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04afd8-a163-4859-9250-7a8cf0c4b27e None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:39.703259 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04afd8-a163-4859-9250-7a8cf0c4b27e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:39.703615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04afd8-a163-4859-9250-7a8cf0c4b27e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:39.705335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04afd8-a163-4859-9250-7a8cf0c4b27e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:39.711134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 479/947] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Sep 14 00:52:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:39.711961 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04afd8-a163-4859-9250-7a8cf0c4b27e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:39.762116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:39.800066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc397a8-2b0d-4005-92ae-1fe6c4a29c0c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:39.822264 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04afd8-a163-4859-9250-7a8cf0c4b27e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:39.841032 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a640c-85cf-49bc-a922-9cbce728f75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:39.841658 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a640c-85cf-49bc-a922-9cbce728f75d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:39.841936 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a640c-85cf-49bc-a922-9cbce728f75d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:39.842205 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a640c-85cf-49bc-a922-9cbce728f75d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:39.849915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a640c-85cf-49bc-a922-9cbce728f75d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:39.851575 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a640c-85cf-49bc-a922-9cbce728f75d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:39.856928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bdacbd-d80b-4576-8ddd-60b72aa48ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:39.857651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bdacbd-d80b-4576-8ddd-60b72aa48ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:39.857916 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bdacbd-d80b-4576-8ddd-60b72aa48ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:39.858636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bdacbd-d80b-4576-8ddd-60b72aa48ad0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:39.863422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-1a1a640c-85cf-49bc-a922-9cbce728f75d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:39.863422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:39.863422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:39.863422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:39.863422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:39.863422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:39.863422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:39.863422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:39.863422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:39.863422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:39.863422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:39.863422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:39.863422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:52:39.863422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:52:39.863422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:52:39.864341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:52:39.864341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:39.864341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:39.864341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:39.864341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:39.864341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:39.864341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:39.864341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:39.864341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:39.864341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:39.864341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:39.864341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:39.864341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:39.864341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:39.864341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:39.864341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:39.864341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:39.865236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:52:39.865236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:52:39.865236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:52:39.865236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:52:39.865236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:52:39.865236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:52:39.865236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:39.865236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:39.865236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 469/948] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:39.865714 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a640c-85cf-49bc-a922-9cbce728f75d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:39.870182 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bdacbd-d80b-4576-8ddd-60b72aa48ad0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:39.871693 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bdacbd-d80b-4576-8ddd-60b72aa48ad0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:39.884553 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-83bdacbd-d80b-4576-8ddd-60b72aa48ad0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:39.884553 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:39.884553 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:39.884553 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:39.884553 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:39.884553 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:39.884553 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:39.884553 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:39.884553 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:39.884553 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:39.884553 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:39.884553 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:39.884553 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:52:39.884553 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:52:39.884553 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:52:39.885502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:52:39.885502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:39.885502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:39.885502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:39.885502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:39.885502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:39.885502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:39.885502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:39.885502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:39.885502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:39.885502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:39.885502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:39.885502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:39.885502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:39.885502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:39.885502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:39.885502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:39.886391 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:52:39.886391 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:52:39.886391 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:52:39.886391 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:52:39.886391 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:52:39.886391 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:52:39.886391 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:39.886391 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:39.886391 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 480/949] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:39.886391 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bdacbd-d80b-4576-8ddd-60b72aa48ad0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:39.924595 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:39.981246 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a640c-85cf-49bc-a922-9cbce728f75d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:40.004564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bdacbd-d80b-4576-8ddd-60b72aa48ad0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:40.008574 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:40.017801 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7714934-4fa6-4860-b6ba-46b05801cca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:40.018365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7714934-4fa6-4860-b6ba-46b05801cca0 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:40.018634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7714934-4fa6-4860-b6ba-46b05801cca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:40.018915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7714934-4fa6-4860-b6ba-46b05801cca0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:40.019654 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7714934-4fa6-4860-b6ba-46b05801cca0 None admin] RBAC: Adding query filter params (interface=public, service_id=c4d1fea9a64b4822951d9245af292a44, region_id=RegionOne) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:40.020012 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7714934-4fa6-4860-b6ba-46b05801cca0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:40.021393 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7714934-4fa6-4860-b6ba-46b05801cca0 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:40.029442 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 470/950] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Sep 14 00:52:39 2022] GET /identity/v3/endpoints?service_id=c4d1fea9a64b4822951d9245af292a44&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:40.030790 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7714934-4fa6-4860-b6ba-46b05801cca0 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:40.050429 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aecbe7-ea4a-4f21-ae81-e2d002cbc729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:40.050944 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aecbe7-ea4a-4f21-ae81-e2d002cbc729 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:40.051228 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aecbe7-ea4a-4f21-ae81-e2d002cbc729 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:40.051511 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aecbe7-ea4a-4f21-ae81-e2d002cbc729 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:40.052252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aecbe7-ea4a-4f21-ae81-e2d002cbc729 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:40.052557 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aecbe7-ea4a-4f21-ae81-e2d002cbc729 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:40.054048 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aecbe7-ea4a-4f21-ae81-e2d002cbc729 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:40.067240 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 481/951] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:52:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:40.072444 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aecbe7-ea4a-4f21-ae81-e2d002cbc729 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:40.142788 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7714934-4fa6-4860-b6ba-46b05801cca0 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:40.156438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:40.178866 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f61885-f82c-47b7-8645-dbed3dc5ddba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:40.179476 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f61885-f82c-47b7-8645-dbed3dc5ddba None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:40.179776 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f61885-f82c-47b7-8645-dbed3dc5ddba None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:40.180045 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f61885-f82c-47b7-8645-dbed3dc5ddba None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:40.180718 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f61885-f82c-47b7-8645-dbed3dc5ddba None admin] RBAC: Adding query filter params (name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:40.181020 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f61885-f82c-47b7-8645-dbed3dc5ddba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:40.182444 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f61885-f82c-47b7-8645-dbed3dc5ddba None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:40.185003 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aecbe7-ea4a-4f21-ae81-e2d002cbc729 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:40.201368 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 471/952] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 00:52:40 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:40.202835 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f61885-f82c-47b7-8645-dbed3dc5ddba None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:40.237422 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96345c-dfa0-476c-b06c-b70e163453c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:40.238054 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96345c-dfa0-476c-b06c-b70e163453c2 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:40.238331 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96345c-dfa0-476c-b06c-b70e163453c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:40.238600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96345c-dfa0-476c-b06c-b70e163453c2 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:40.239801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96345c-dfa0-476c-b06c-b70e163453c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:40.241623 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96345c-dfa0-476c-b06c-b70e163453c2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:40.249315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 482/953] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Sep 14 00:52:40 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 00:52:40.332733 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f61885-f82c-47b7-8645-dbed3dc5ddba None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:40.375801 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c41b14-a055-4e9d-a7a1-d823dc29a7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:40.376990 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c41b14-a055-4e9d-a7a1-d823dc29a7e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:40.377485 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c41b14-a055-4e9d-a7a1-d823dc29a7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:40.377989 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c41b14-a055-4e9d-a7a1-d823dc29a7e5 None admin] PATH_INFO: `/v3/projects/fda69e8c2d5e466dbc1244b2f76296a3/users/ceccc909529042539dd2d5015058d8f7/roles/ae888494799042399af96f094898c4ed` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:40.380844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c41b14-a055-4e9d-a7a1-d823dc29a7e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=fda69e8c2d5e466dbc1244b2f76296a3, user_id=ceccc909529042539dd2d5015058d8f7, role_id=ae888494799042399af96f094898c4ed)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:40.382737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c41b14-a055-4e9d-a7a1-d823dc29a7e5 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:40.403431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 472/954] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 00:52:40 2022] PUT /identity/v3/projects/fda69e8c2d5e466dbc1244b2f76296a3/users/ceccc909529042539dd2d5015058d8f7/roles/ae888494799042399af96f094898c4ed => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:52:41.009279 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1140d-d715-4ade-9e26-2c619d72ef23 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:41.009805 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1140d-d715-4ade-9e26-2c619d72ef23 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:41.009805 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea1140d-d715-4ade-9e26-2c619d72ef23 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:41.011258 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 483/955] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:41.017297 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db7840-397c-4a75-b02d-4b0ba99014b7 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:41.017740 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db7840-397c-4a75-b02d-4b0ba99014b7 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:41.017942 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db7840-397c-4a75-b02d-4b0ba99014b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:41.037730 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a8db7840-397c-4a75-b02d-4b0ba99014b7 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:41.039235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a8db7840-397c-4a75-b02d-4b0ba99014b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:41.067456 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:41.115232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 473/956] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:41.385845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1140d-d715-4ade-9e26-2c619d72ef23 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:41.416418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:41.491771 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea1140d-d715-4ade-9e26-2c619d72ef23 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:41.526958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc55b83-69fb-481f-8f2f-245d52299227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:41.527585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc55b83-69fb-481f-8f2f-245d52299227 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:41.527874 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc55b83-69fb-481f-8f2f-245d52299227 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:41.528151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc55b83-69fb-481f-8f2f-245d52299227 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:41.528807 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc55b83-69fb-481f-8f2f-245d52299227 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:41.530751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc55b83-69fb-481f-8f2f-245d52299227 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:41.547747 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 484/957] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 00:52:41 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:52:42.552798 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dca719-1694-47d9-ba68-ff111bf6bbc8 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:42.553810 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dca719-1694-47d9-ba68-ff111bf6bbc8 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:42.554345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dca719-1694-47d9-ba68-ff111bf6bbc8 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:42.556354 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 474/958] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:42.558014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89bd88-5f3a-4f73-9fd3-69e16cd1bb9b None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:42.558501 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89bd88-5f3a-4f73-9fd3-69e16cd1bb9b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:42.561472 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89bd88-5f3a-4f73-9fd3-69e16cd1bb9b None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:42.561472 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 485/959] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:52:42.563458 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd04af3-f0aa-4255-a579-e661fbe45a1b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:42.563783 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd04af3-f0aa-4255-a579-e661fbe45a1b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:42.564058 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd04af3-f0aa-4255-a579-e661fbe45a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:42.568195 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d8712-18c9-4295-947d-36e3d1f685b4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:42.568480 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d8712-18c9-4295-947d-36e3d1f685b4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:42.568747 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d8712-18c9-4295-947d-36e3d1f685b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:42.590390 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-0fd04af3-f0aa-4255-a579-e661fbe45a1b None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:42.591604 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0fd04af3-f0aa-4255-a579-e661fbe45a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:42.592129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-945d8712-18c9-4295-947d-36e3d1f685b4 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:42.594087 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-945d8712-18c9-4295-947d-36e3d1f685b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:42.677874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 475/960] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:42 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:42.678980 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc718d7-c8e2-45b8-974c-bb5115231689 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:42.679269 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc718d7-c8e2-45b8-974c-bb5115231689 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:42.679609 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc718d7-c8e2-45b8-974c-bb5115231689 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:42.679924 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 486/961] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:42 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:42.681047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 476/962] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:42.685966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb158f0a-de48-4aa9-8e77-1beb9daf90a2 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:42.686320 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb158f0a-de48-4aa9-8e77-1beb9daf90a2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:42.686589 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb158f0a-de48-4aa9-8e77-1beb9daf90a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:42.705196 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-eb158f0a-de48-4aa9-8e77-1beb9daf90a2 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:42.706382 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-eb158f0a-de48-4aa9-8e77-1beb9daf90a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:42.720109 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:42.781823 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 487/963] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:42 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 00:52:42.955577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc718d7-c8e2-45b8-974c-bb5115231689 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:42.974238 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb158f0a-de48-4aa9-8e77-1beb9daf90a2 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:43.062519 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:43.079331 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc718d7-c8e2-45b8-974c-bb5115231689 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:43.101820 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb158f0a-de48-4aa9-8e77-1beb9daf90a2 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:43.116109 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515a8d1-e2e1-48a4-96c9-88b3b6b45118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:43.116750 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515a8d1-e2e1-48a4-96c9-88b3b6b45118 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:43.117030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515a8d1-e2e1-48a4-96c9-88b3b6b45118 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:43.117301 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515a8d1-e2e1-48a4-96c9-88b3b6b45118 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:43.117888 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9515a8d1-e2e1-48a4-96c9-88b3b6b45118 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:43.119265 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9515a8d1-e2e1-48a4-96c9-88b3b6b45118 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:43.128538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-9515a8d1-e2e1-48a4-96c9-88b3b6b45118 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 00:52:43.128538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:43.128538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:52:43.128538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:52:43.128538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:52:43.128538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:52:43.128538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:52:43.128538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:43.128538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 00:52:43.128538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:43.128538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:43.128538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:43.128538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:43.128538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:43.128538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:43.128538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:43.128538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:43.135181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:43.135181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:43.135181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:43.135181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:43.135181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:43.135181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:43.135181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:52:43.135181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:52:43.135181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:52:43.135181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:52:43.135181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:43.135181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:43.135181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:43.135181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:43.135181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:43.135181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:43.136333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:43.136333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:43.136333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:43.136333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:43.136333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:43.136333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:43.136333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:43.136333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:43.136333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:43.136333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:43.136333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:52:43.136333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:43.136333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 00:52:43.136333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:43.136333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 477/964] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Sep 14 00:52:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:43.137353 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515a8d1-e2e1-48a4-96c9-88b3b6b45118 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:43.150659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e41f6-67b0-4607-b270-ed8ae409df19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:43.151467 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e41f6-67b0-4607-b270-ed8ae409df19 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:43.151923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e41f6-67b0-4607-b270-ed8ae409df19 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:43.152359 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e41f6-67b0-4607-b270-ed8ae409df19 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:43.153290 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12e41f6-67b0-4607-b270-ed8ae409df19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:43.154716 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12e41f6-67b0-4607-b270-ed8ae409df19 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:43.157782 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: WARNING keystone.server.flask.common [None req-c12e41f6-67b0-4607-b270-ed8ae409df19 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 14 00:52:43.172739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.sql.core [None req-c12e41f6-67b0-4607-b270-ed8ae409df19 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 14 00:52:43.172739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 14 00:52:43.172739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [parameters: {'user_id': 'dd11d9eb118c45faa39b75310bbba15b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 14 00:52:43.172739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81417) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 14 00:52:43.190919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-c12e41f6-67b0-4607-b270-ed8ae409df19 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 14 00:52:43.190919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:43.190919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 00:52:43.190919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 00:52:43.190919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 00:52:43.190919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 00:52:43.190919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 00:52:43.190919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 00:52:43.190919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 00:52:43.190919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application conn.query(q) Sep 14 00:52:43.190919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 00:52:43.190919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 00:52:43.190919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 00:52:43.190919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application result.read() Sep 14 00:52:43.190919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 00:52:43.191870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 00:52:43.191870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 00:52:43.191870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 00:52:43.191870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 00:52:43.191870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 00:52:43.191870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 00:52:43.191870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 00:52:43.191870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 14 00:52:43.191870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:43.191870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 14 00:52:43.191870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:43.191870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:43.191870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 14 00:52:43.191870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 14 00:52:43.191870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 14 00:52:43.191870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 14 00:52:43.191870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 00:52:43.191870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application next(self.gen) Sep 14 00:52:43.197870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 14 00:52:43.197870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application yield resource Sep 14 00:52:43.197870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 00:52:43.197870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application next(self.gen) Sep 14 00:52:43.197870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 14 00:52:43.197870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self.session.rollback() Sep 14 00:52:43.197870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 14 00:52:43.197870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self.force_reraise() Sep 14 00:52:43.197870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 14 00:52:43.197870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise self.value Sep 14 00:52:43.197870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 14 00:52:43.197870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 14 00:52:43.197870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 14 00:52:43.197870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application session.commit() Sep 14 00:52:43.197870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 14 00:52:43.197870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 14 00:52:43.197870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 14 00:52:43.198790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._prepare_impl() Sep 14 00:52:43.198790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 14 00:52:43.198790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self.session.flush() Sep 14 00:52:43.198790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 14 00:52:43.198790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._flush(objects) Sep 14 00:52:43.198790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 14 00:52:43.198790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 14 00:52:43.198790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 14 00:52:43.198790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application compat.raise_( Sep 14 00:52:43.198790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 00:52:43.198790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception Sep 14 00:52:43.198790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 14 00:52:43.198790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application flush_context.execute() Sep 14 00:52:43.198790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 14 00:52:43.198790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rec.execute(self) Sep 14 00:52:43.198790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 14 00:52:43.198790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 14 00:52:43.199728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 14 00:52:43.199728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 14 00:52:43.199728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 14 00:52:43.199728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 14 00:52:43.199728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 14 00:52:43.199728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 14 00:52:43.199728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 14 00:52:43.199728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 14 00:52:43.199728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 14 00:52:43.199728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 14 00:52:43.199728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 14 00:52:43.199728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 14 00:52:43.199728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 14 00:52:43.199728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 14 00:52:43.199728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 00:52:43.199728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception Sep 14 00:52:43.200604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 00:52:43.200604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 00:52:43.200604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 00:52:43.200604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 00:52:43.200604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 00:52:43.200604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 00:52:43.200604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 00:52:43.200604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application conn.query(q) Sep 14 00:52:43.200604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 00:52:43.200604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 00:52:43.200604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 00:52:43.200604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application result.read() Sep 14 00:52:43.200604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 00:52:43.200604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 00:52:43.200604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 00:52:43.200604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 00:52:43.201449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 00:52:43.201449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 00:52:43.201449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 00:52:43.201449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 00:52:43.201449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 14 00:52:43.201449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 14 00:52:43.201449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [parameters: {'user_id': 'dd11d9eb118c45faa39b75310bbba15b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 14 00:52:43.201449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 14 00:52:43.201449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:43.201449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:43.201449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:43.201449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:43.201449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:43.201449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:43.201449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:43.203356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:43.203356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:43.203356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:43.203356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:43.203356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:43.203356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:43.203356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:43.203356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 14 00:52:43.203356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 14 00:52:43.203356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:43.203356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:43.203356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:52:43.203356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:43.203356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:52:43.203356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:43.203356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 14 00:52:43.203356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 14 00:52:43.204443 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 14 00:52:43.204443 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 14 00:52:43.204443 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 14 00:52:43.204443 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 14 00:52:43.204443 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 14 00:52:43.204443 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:43.204443 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 488/965] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Sep 14 00:52:42 2022] POST /identity/v3/users => generated 160 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:43.204443 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e41f6-67b0-4607-b270-ed8ae409df19 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:43.214972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:43.249085 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515a8d1-e2e1-48a4-96c9-88b3b6b45118 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:43.286570 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d5ba68-be1a-455b-9f1b-62a402b52c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:43.287537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5ba68-be1a-455b-9f1b-62a402b52c56 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:43.288008 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5ba68-be1a-455b-9f1b-62a402b52c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:43.288520 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5ba68-be1a-455b-9f1b-62a402b52c56 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:43.299983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d5ba68-be1a-455b-9f1b-62a402b52c56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:52:43.305891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-51d5ba68-be1a-455b-9f1b-62a402b52c56 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 00:52:43.305891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:43.305891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:43.305891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:43.305891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:43.305891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:43.305891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:43.305891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:43.305891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:43.305891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:43.305891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:43.305891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:43.305891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:52:43.305891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:52:43.305891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:52:43.305891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:52:43.307030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:52:43.307030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:52:43.307030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:52:43.307030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:52:43.307030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:43.307030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:43.307030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:43.307030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:43.307030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:43.307030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:43.307030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:43.307030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:43.307030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:43.307030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:43.307030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:43.307030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:43.308077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:43.308077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:43.308077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:43.308077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:43.308077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:52:43.308077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:52:43.308077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:52:43.308077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:52:43.308077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:52:43.308077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:52:43.308077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 00:52:43.308077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:43.310509 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 478/966] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Sep 14 00:52:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:43.311150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e41f6-67b0-4607-b270-ed8ae409df19 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:43.312033 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d5ba68-be1a-455b-9f1b-62a402b52c56 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:43.329005 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:43.352044 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d55336-0ebd-4ebb-a649-5018be5fd9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:43.352657 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d55336-0ebd-4ebb-a649-5018be5fd9e1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:43.352937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d55336-0ebd-4ebb-a649-5018be5fd9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:43.353207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d55336-0ebd-4ebb-a649-5018be5fd9e1 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:43.353831 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d55336-0ebd-4ebb-a649-5018be5fd9e1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:43.354131 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d55336-0ebd-4ebb-a649-5018be5fd9e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:43.355747 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d55336-0ebd-4ebb-a649-5018be5fd9e1 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:43.367612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 489/967] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Sep 14 00:52:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:43.367612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d55336-0ebd-4ebb-a649-5018be5fd9e1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:43.421557 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d5ba68-be1a-455b-9f1b-62a402b52c56 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:43.448934 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:43.458551 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f1efd-8c45-493f-bab6-3582dd93cfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:43.459503 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f1efd-8c45-493f-bab6-3582dd93cfdf None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:43.460050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f1efd-8c45-493f-bab6-3582dd93cfdf None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:43.460514 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f1efd-8c45-493f-bab6-3582dd93cfdf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:43.474385 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f1efd-8c45-493f-bab6-3582dd93cfdf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:43.474821 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d55336-0ebd-4ebb-a649-5018be5fd9e1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:43.476327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f1efd-8c45-493f-bab6-3582dd93cfdf None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:43.491402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-d20f1efd-8c45-493f-bab6-3582dd93cfdf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 00:52:43.491402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:43.491402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:43.491402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:43.491402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:43.491402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:43.491402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:43.491402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:43.491402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:43.491402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:43.491402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:43.491402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:43.491402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:52:43.491402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:52:43.491402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:52:43.491402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:52:43.492443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:43.492443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:43.492443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:52:43.492443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:43.492443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:52:43.492443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:43.492443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:43.492443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:43.492443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:43.492443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:43.492443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:43.492443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:43.492443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:43.492443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:43.492443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:43.492443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:43.492443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:43.492443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:43.493406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:43.493406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:43.493406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:52:43.493406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:52:43.493406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:52:43.493406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:52:43.493406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:52:43.493406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:52:43.493406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 00:52:43.493406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:43.493406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 479/968] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:52:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:43.493406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f1efd-8c45-493f-bab6-3582dd93cfdf None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:43.522631 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9cae3-eb4c-443f-9eba-b96dba918749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:43.523319 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9cae3-eb4c-443f-9eba-b96dba918749 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:43.523631 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9cae3-eb4c-443f-9eba-b96dba918749 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:43.523913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd9cae3-eb4c-443f-9eba-b96dba918749 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:43.524562 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9cae3-eb4c-443f-9eba-b96dba918749 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:43.524863 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9cae3-eb4c-443f-9eba-b96dba918749 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:43.526218 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd9cae3-eb4c-443f-9eba-b96dba918749 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:43.539234 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 490/969] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Sep 14 00:52:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:43.539234 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9cae3-eb4c-443f-9eba-b96dba918749 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:43.577812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:43.610191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f1efd-8c45-493f-bab6-3582dd93cfdf None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:43.641682 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd9cae3-eb4c-443f-9eba-b96dba918749 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:43.647804 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b84cf-5f58-4132-b0bf-8330b4763508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:43.648444 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b84cf-5f58-4132-b0bf-8330b4763508 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:43.648726 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b84cf-5f58-4132-b0bf-8330b4763508 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:43.649002 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b84cf-5f58-4132-b0bf-8330b4763508 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:43.649666 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4b84cf-5f58-4132-b0bf-8330b4763508 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:43.649979 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4b84cf-5f58-4132-b0bf-8330b4763508 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:43.651401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4b84cf-5f58-4132-b0bf-8330b4763508 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:43.662832 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 480/970] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Sep 14 00:52:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:43.662832 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b84cf-5f58-4132-b0bf-8330b4763508 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:43.682300 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a3d0e-76e9-46b8-85ff-94cc659d7efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:43.682913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a3d0e-76e9-46b8-85ff-94cc659d7efb None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:43.683194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a3d0e-76e9-46b8-85ff-94cc659d7efb None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:43.683471 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a3d0e-76e9-46b8-85ff-94cc659d7efb None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:43.684194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a3d0e-76e9-46b8-85ff-94cc659d7efb None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:43.684497 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a3d0e-76e9-46b8-85ff-94cc659d7efb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:43.685921 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a3d0e-76e9-46b8-85ff-94cc659d7efb None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:43.698102 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:43.731607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 491/971] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 00:52:43 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:43.731607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a3d0e-76e9-46b8-85ff-94cc659d7efb None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:43.782440 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b84cf-5f58-4132-b0bf-8330b4763508 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:43.820932 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064202e6-f173-439b-bf46-18450304087e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:43.821590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064202e6-f173-439b-bf46-18450304087e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:43.821949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064202e6-f173-439b-bf46-18450304087e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:43.822147 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064202e6-f173-439b-bf46-18450304087e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:43.825596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:43.835676 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064202e6-f173-439b-bf46-18450304087e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:43.837175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064202e6-f173-439b-bf46-18450304087e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:43.854103 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-064202e6-f173-439b-bf46-18450304087e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:43.854103 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:43.854103 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:43.854103 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:43.854103 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:43.854103 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:43.854103 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:43.854103 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:43.854103 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:43.854103 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:43.854103 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:43.854103 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:43.854103 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:43.854103 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:43.854103 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:43.854103 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:43.855139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:43.855139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:43.855139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:43.855139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:43.855139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:43.855139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:43.855139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:43.855139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:43.855139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:43.855139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:43.855139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:43.855139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:43.855139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:43.855139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:43.855139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:43.855139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:43.855139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:43.856661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:43.856661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:43.856661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:43.856839 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a3d0e-76e9-46b8-85ff-94cc659d7efb None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:43.861054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 481/972] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:43.862290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714ddf0-705b-46e4-a76a-43dee54f251f None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:43.862586 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714ddf0-705b-46e4-a76a-43dee54f251f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:43.862869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714ddf0-705b-46e4-a76a-43dee54f251f None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:43.864539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 482/973] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:43.865071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1714ddf0-705b-46e4-a76a-43dee54f251f None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:43.900355 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c37cf2-e019-4253-bc16-66e9a57e786e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:43.900977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c37cf2-e019-4253-bc16-66e9a57e786e None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:43.901260 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c37cf2-e019-4253-bc16-66e9a57e786e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:43.901537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c37cf2-e019-4253-bc16-66e9a57e786e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:43.902211 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c37cf2-e019-4253-bc16-66e9a57e786e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:43.903687 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c37cf2-e019-4253-bc16-66e9a57e786e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:43.931762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 492/974] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 00:52:43 2022] POST /identity/v3/endpoints => generated 349 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:52:43.933096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724f531-f743-4f8f-8c83-45c761906071 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:43.933386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724f531-f743-4f8f-8c83-45c761906071 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:43.933684 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724f531-f743-4f8f-8c83-45c761906071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:43.959837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-6724f531-f743-4f8f-8c83-45c761906071 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:43.961451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6724f531-f743-4f8f-8c83-45c761906071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:44.002341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1714ddf0-705b-46e4-a76a-43dee54f251f None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:44.026679 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:44.054299 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df9e5d-62bb-4bf7-acc7-69dda16850d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:44.055326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df9e5d-62bb-4bf7-acc7-69dda16850d4 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:44.055705 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df9e5d-62bb-4bf7-acc7-69dda16850d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:44.055983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df9e5d-62bb-4bf7-acc7-69dda16850d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:44.056650 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df9e5d-62bb-4bf7-acc7-69dda16850d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:44.056956 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df9e5d-62bb-4bf7-acc7-69dda16850d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:44.058666 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df9e5d-62bb-4bf7-acc7-69dda16850d4 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:44.068915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 493/975] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:43 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:44.075503 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 483/976] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:44.078133 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724f531-f743-4f8f-8c83-45c761906071 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:44.185681 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724f531-f743-4f8f-8c83-45c761906071 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:44.221012 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2414356-78fd-4694-88a6-6b1f01b3505b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:44.221647 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2414356-78fd-4694-88a6-6b1f01b3505b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:44.221926 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2414356-78fd-4694-88a6-6b1f01b3505b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:44.222195 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2414356-78fd-4694-88a6-6b1f01b3505b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:44.229368 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2414356-78fd-4694-88a6-6b1f01b3505b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:44.230802 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2414356-78fd-4694-88a6-6b1f01b3505b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:44.242838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-f2414356-78fd-4694-88a6-6b1f01b3505b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 00:52:44.242838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:44.242838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:44.242838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:44.242838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:44.242838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:44.242838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:44.242838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:44.242838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:44.242838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:44.242838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:44.242838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:44.242838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:52:44.242838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:52:44.242838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:52:44.242838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:52:44.244145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:44.244145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:44.244145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:52:44.244145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:44.244145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:52:44.244145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:44.244145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:44.244145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:44.244145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:44.244145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:44.244145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:44.244145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:44.244145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:44.244145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:44.244145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:44.244145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:44.244145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:44.244145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:44.245163 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:44.245163 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:44.245163 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:52:44.245163 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:52:44.245163 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:52:44.245163 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:52:44.245163 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:52:44.245163 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:52:44.245163 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 00:52:44.245163 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:44.247615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 494/977] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Sep 14 00:52:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:44.250615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df9e5d-62bb-4bf7-acc7-69dda16850d4 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:44.298283 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:44.353678 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2414356-78fd-4694-88a6-6b1f01b3505b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:44.356078 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df9e5d-62bb-4bf7-acc7-69dda16850d4 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:44.394254 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfdc74-7efb-4c64-8a12-783130b77f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:44.394866 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfdc74-7efb-4c64-8a12-783130b77f48 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:44.395147 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfdc74-7efb-4c64-8a12-783130b77f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:44.395424 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfdc74-7efb-4c64-8a12-783130b77f48 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:44.396184 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edfdc74-7efb-4c64-8a12-783130b77f48 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:44.396490 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edfdc74-7efb-4c64-8a12-783130b77f48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:44.397934 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edfdc74-7efb-4c64-8a12-783130b77f48 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:44.399248 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:44.417632 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 484/978] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Sep 14 00:52:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:44.423982 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfdc74-7efb-4c64-8a12-783130b77f48 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:44.468156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2414356-78fd-4694-88a6-6b1f01b3505b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:44.510317 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2af13d-b187-40db-a079-259e46f6fefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:44.511114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2af13d-b187-40db-a079-259e46f6fefa None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:44.511400 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2af13d-b187-40db-a079-259e46f6fefa None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:44.511766 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2af13d-b187-40db-a079-259e46f6fefa None admin] PATH_INFO: `/v3/services/compute` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:44.512355 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2af13d-b187-40db-a079-259e46f6fefa None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:44.513798 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2af13d-b187-40db-a079-259e46f6fefa None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:44.523392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-6f2af13d-b187-40db-a079-259e46f6fefa None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 00:52:44.523392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:44.523392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:52:44.523392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:52:44.523392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:52:44.523392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:52:44.523392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:52:44.523392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:44.523392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 00:52:44.523392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:44.523392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:44.523392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:44.523392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:44.523392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:44.523392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:44.523392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:44.523392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:44.524455 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:44.524455 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:44.524455 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:44.524455 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:44.524455 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:44.524455 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:44.524455 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:52:44.524455 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:52:44.524455 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:52:44.524455 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:52:44.524455 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:44.524455 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:44.524455 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:44.524455 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:44.524455 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:44.524455 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:44.525316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:44.525316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:44.525316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:44.525316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:44.525316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:44.525316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:44.525316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:44.525316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:44.525316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:44.525316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:44.525316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:52:44.525316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:44.525316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 00:52:44.525316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:44.525316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 495/979] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:52:44.530998 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2af13d-b187-40db-a079-259e46f6fefa None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:44.535169 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfdc74-7efb-4c64-8a12-783130b77f48 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:44.536765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:44.571730 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e21ef4b-a31c-4e9e-93e6-3d9af788b3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:44.572347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21ef4b-a31c-4e9e-93e6-3d9af788b3c6 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:44.572627 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21ef4b-a31c-4e9e-93e6-3d9af788b3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:44.572899 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21ef4b-a31c-4e9e-93e6-3d9af788b3c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:44.637917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2af13d-b187-40db-a079-259e46f6fefa None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:44.643929 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:44.648418 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e21ef4b-a31c-4e9e-93e6-3d9af788b3c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:44.650361 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e21ef4b-a31c-4e9e-93e6-3d9af788b3c6 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:44.678146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b40d5-56bd-487b-90ef-60fbe4fc1308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:44.678759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b40d5-56bd-487b-90ef-60fbe4fc1308 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:44.679039 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b40d5-56bd-487b-90ef-60fbe4fc1308 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:44.679311 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b40d5-56bd-487b-90ef-60fbe4fc1308 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:44.679972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b40d5-56bd-487b-90ef-60fbe4fc1308 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:44.680277 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b40d5-56bd-487b-90ef-60fbe4fc1308 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:44.681639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b40d5-56bd-487b-90ef-60fbe4fc1308 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:44.687311 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 496/980] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 00:52:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:44.692855 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b40d5-56bd-487b-90ef-60fbe4fc1308 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:44.754652 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 485/981] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:52:44 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 14 00:52:44.760322 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e21ef4b-a31c-4e9e-93e6-3d9af788b3c6 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:44.762538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:44.810995 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b40d5-56bd-487b-90ef-60fbe4fc1308 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:44.854628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593e260-13e5-44f7-89fd-2320d7550cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:44.855549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593e260-13e5-44f7-89fd-2320d7550cdc None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:44.855917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593e260-13e5-44f7-89fd-2320d7550cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:44.856196 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593e260-13e5-44f7-89fd-2320d7550cdc None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:44.856881 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593e260-13e5-44f7-89fd-2320d7550cdc None admin] RBAC: Adding query filter params (type=compute) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:44.857191 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593e260-13e5-44f7-89fd-2320d7550cdc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:44.858662 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593e260-13e5-44f7-89fd-2320d7550cdc None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:44.869737 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 497/982] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 00:52:44 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:44.869935 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e21ef4b-a31c-4e9e-93e6-3d9af788b3c6 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:44.870785 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593e260-13e5-44f7-89fd-2320d7550cdc None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:44.902333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b70287-74f7-4a4c-9132-e673cde5a329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:44.902963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b70287-74f7-4a4c-9132-e673cde5a329 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:44.903242 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b70287-74f7-4a4c-9132-e673cde5a329 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:44.903520 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b70287-74f7-4a4c-9132-e673cde5a329 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:44.908817 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:44.913800 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b70287-74f7-4a4c-9132-e673cde5a329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:44.915253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b70287-74f7-4a4c-9132-e673cde5a329 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:44.934291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-f8b70287-74f7-4a4c-9132-e673cde5a329 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:44.934291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:44.934291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:44.934291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:44.934291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:44.934291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:44.934291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:44.934291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:44.934291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:44.934291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:44.934291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:44.934291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:44.934291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:44.934291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:44.934291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:44.934291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:44.935291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:44.935291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:44.935291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:44.935291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:44.935291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:44.935291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:44.935291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:44.935291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:44.935291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:44.935291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:44.935291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:44.935291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:44.935291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:44.935291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:44.935291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:44.935291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:44.935291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:44.936830 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:44.936830 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:44.936830 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:44.936830 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 486/983] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:52:44.942099 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b70287-74f7-4a4c-9132-e673cde5a329 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:44.990114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593e260-13e5-44f7-89fd-2320d7550cdc None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:45.001544 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:45.029353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c61e4-4a5e-4a15-ab7e-e7bf13df6d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:45.030054 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c61e4-4a5e-4a15-ab7e-e7bf13df6d26 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:45.030444 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c61e4-4a5e-4a15-ab7e-e7bf13df6d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:45.030716 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c61e4-4a5e-4a15-ab7e-e7bf13df6d26 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:45.031417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c61e4-4a5e-4a15-ab7e-e7bf13df6d26 None admin] RBAC: Adding query filter params (interface=public, service_id=c52a67d8408441cc81fdbc9bb6ecfdd3, region_id=RegionOne) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:45.031770 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c61e4-4a5e-4a15-ab7e-e7bf13df6d26 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:45.033135 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c61e4-4a5e-4a15-ab7e-e7bf13df6d26 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:45.038233 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 498/984] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Sep 14 00:52:44 2022] GET /identity/v3/endpoints?service_id=c52a67d8408441cc81fdbc9bb6ecfdd3&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:45.043847 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c61e4-4a5e-4a15-ab7e-e7bf13df6d26 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:45.050370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b70287-74f7-4a4c-9132-e673cde5a329 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:45.085996 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d426202-8a4e-4936-b02e-fa0d47c7412a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:45.086730 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d426202-8a4e-4936-b02e-fa0d47c7412a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:45.087015 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d426202-8a4e-4936-b02e-fa0d47c7412a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:45.087289 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d426202-8a4e-4936-b02e-fa0d47c7412a None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:45.087967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d426202-8a4e-4936-b02e-fa0d47c7412a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:45.088273 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d426202-8a4e-4936-b02e-fa0d47c7412a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:45.089800 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d426202-8a4e-4936-b02e-fa0d47c7412a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:45.103050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 487/985] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:45.108481 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d426202-8a4e-4936-b02e-fa0d47c7412a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:45.149887 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:45.158034 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c61e4-4a5e-4a15-ab7e-e7bf13df6d26 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:45.197124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b159a-7fa9-4684-8848-ca2880df4ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:45.197744 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b159a-7fa9-4684-8848-ca2880df4ceb None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:45.198033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b159a-7fa9-4684-8848-ca2880df4ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:45.198300 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b159a-7fa9-4684-8848-ca2880df4ceb None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:45.198915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b159a-7fa9-4684-8848-ca2880df4ceb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:45.200312 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b159a-7fa9-4684-8848-ca2880df4ceb None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:45.207031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 499/986] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Sep 14 00:52:45 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 00:52:45.219956 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d426202-8a4e-4936-b02e-fa0d47c7412a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:45.236785 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:45.260535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3d5c3-6792-472f-8dd3-5c68cb686655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:45.261202 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3d5c3-6792-472f-8dd3-5c68cb686655 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:45.261484 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3d5c3-6792-472f-8dd3-5c68cb686655 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:45.261756 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3d5c3-6792-472f-8dd3-5c68cb686655 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:45.270073 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3d5c3-6792-472f-8dd3-5c68cb686655 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:45.271598 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3d5c3-6792-472f-8dd3-5c68cb686655 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:45.284615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-c9d3d5c3-6792-472f-8dd3-5c68cb686655 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:45.284615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:45.284615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:45.284615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:45.284615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:45.284615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:45.284615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:45.284615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:45.284615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:45.284615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:45.284615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:45.284615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:45.284615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:52:45.284615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:52:45.284615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:52:45.285589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:52:45.285589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:45.285589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:45.285589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:45.285589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:45.285589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:45.285589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:45.285589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:45.285589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:45.285589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:45.285589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:45.285589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:45.285589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:45.285589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:45.285589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:45.285589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:45.285589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:45.291936 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:52:45.291936 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:52:45.291936 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:52:45.291936 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:52:45.291936 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:52:45.291936 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:52:45.291936 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:45.291936 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:45.291936 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 488/987] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:52:45.292442 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b159a-7fa9-4684-8848-ca2880df4ceb None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:45.400917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b159a-7fa9-4684-8848-ca2880df4ceb None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:45.437474 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab79f82-cd5f-4e24-92c8-4ca40586fd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:45.438062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab79f82-cd5f-4e24-92c8-4ca40586fd7f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:45.438339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab79f82-cd5f-4e24-92c8-4ca40586fd7f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:45.438611 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab79f82-cd5f-4e24-92c8-4ca40586fd7f None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:45.439290 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab79f82-cd5f-4e24-92c8-4ca40586fd7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:45.439617 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab79f82-cd5f-4e24-92c8-4ca40586fd7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:45.441108 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab79f82-cd5f-4e24-92c8-4ca40586fd7f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:45.453272 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 500/988] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:52:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:45.459141 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3d5c3-6792-472f-8dd3-5c68cb686655 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:45.567818 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3d5c3-6792-472f-8dd3-5c68cb686655 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:45.602454 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5bc190-70e8-467c-807c-4caf4036692f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:45.603430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bc190-70e8-467c-807c-4caf4036692f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:45.603958 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bc190-70e8-467c-807c-4caf4036692f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:45.604454 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bc190-70e8-467c-807c-4caf4036692f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:45.605488 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5bc190-70e8-467c-807c-4caf4036692f None admin] RBAC: Adding query filter params (role.id=ae888494799042399af96f094898c4ed, scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3, user.id=ceccc909529042539dd2d5015058d8f7) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:45.605986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5bc190-70e8-467c-807c-4caf4036692f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:45.607608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5bc190-70e8-467c-807c-4caf4036692f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:45.614734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 489/989] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:52:45 2022] GET /identity/v3/role_assignments?user.id=ceccc909529042539dd2d5015058d8f7&scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3&role.id=ae888494799042399af96f094898c4ed => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:46.150431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd802b-712a-493b-9eda-e9a9bb2775d9 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:46.150998 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd802b-712a-493b-9eda-e9a9bb2775d9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:46.150998 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd802b-712a-493b-9eda-e9a9bb2775d9 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:46.152474 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 501/990] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:46.157850 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5a0c3-06ae-465f-8ef8-6b4743872369 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:46.158246 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5a0c3-06ae-465f-8ef8-6b4743872369 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:46.158445 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5a0c3-06ae-465f-8ef8-6b4743872369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:46.176620 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-c0e5a0c3-06ae-465f-8ef8-6b4743872369 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:46.177815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c0e5a0c3-06ae-465f-8ef8-6b4743872369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:46.209219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:46.248743 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 490/991] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:46 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:46.513475 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd802b-712a-493b-9eda-e9a9bb2775d9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:46.538787 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:46.618536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd802b-712a-493b-9eda-e9a9bb2775d9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:46.651292 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354fe30-8049-41c3-907f-3fd665624d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:46.651940 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354fe30-8049-41c3-907f-3fd665624d22 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:46.652220 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354fe30-8049-41c3-907f-3fd665624d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:46.652491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354fe30-8049-41c3-907f-3fd665624d22 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:46.653062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7354fe30-8049-41c3-907f-3fd665624d22 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:46.654416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7354fe30-8049-41c3-907f-3fd665624d22 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:46.663710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-7354fe30-8049-41c3-907f-3fd665624d22 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 00:52:46.663710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:46.663710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:52:46.663710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:52:46.663710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:52:46.663710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:52:46.663710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:52:46.663710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:46.663710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 00:52:46.663710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:46.663710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:46.663710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:46.663710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:46.663710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:46.663710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:46.663710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:46.663710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:46.664724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:46.664724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:46.664724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:46.664724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:46.664724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:46.664724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:46.664724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:52:46.664724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:52:46.664724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:52:46.664724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:52:46.664724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:46.664724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:46.664724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:46.664724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:46.664724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:46.664724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:46.665570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:46.665570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:46.665570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:46.665570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:46.665570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:46.665570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:46.665570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:46.665570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:46.665570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:46.665570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:46.665570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:52:46.665570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:46.665570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 00:52:46.665570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:46.667640 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 502/992] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Sep 14 00:52:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:46.671251 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5a0c3-06ae-465f-8ef8-6b4743872369 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:46.771250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5a0c3-06ae-465f-8ef8-6b4743872369 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:46.804857 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffcd9b5-e176-4a14-ae29-44e33478a93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:46.805469 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcd9b5-e176-4a14-ae29-44e33478a93d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:46.805749 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcd9b5-e176-4a14-ae29-44e33478a93d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:46.806022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcd9b5-e176-4a14-ae29-44e33478a93d None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:46.806678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffcd9b5-e176-4a14-ae29-44e33478a93d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:46.806983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffcd9b5-e176-4a14-ae29-44e33478a93d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:46.808378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffcd9b5-e176-4a14-ae29-44e33478a93d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:46.818872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 491/993] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Sep 14 00:52:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:46.820478 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354fe30-8049-41c3-907f-3fd665624d22 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:46.907925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:46.924940 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354fe30-8049-41c3-907f-3fd665624d22 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:46.959474 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c36e2-9bc4-4a7f-abc7-84df1b189cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:46.960126 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c36e2-9bc4-4a7f-abc7-84df1b189cae None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:46.960404 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c36e2-9bc4-4a7f-abc7-84df1b189cae None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:46.960681 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c36e2-9bc4-4a7f-abc7-84df1b189cae None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:46.961305 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c36e2-9bc4-4a7f-abc7-84df1b189cae None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:46.961606 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c36e2-9bc4-4a7f-abc7-84df1b189cae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:46.962966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c36e2-9bc4-4a7f-abc7-84df1b189cae None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:46.972669 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 503/994] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Sep 14 00:52:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:46.973964 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffcd9b5-e176-4a14-ae29-44e33478a93d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:47.007956 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:47.080625 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffcd9b5-e176-4a14-ae29-44e33478a93d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:47.115050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b32b3c-fb72-4b5b-bfc4-a187f7153a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:47.115695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b32b3c-fb72-4b5b-bfc4-a187f7153a7c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:47.115976 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b32b3c-fb72-4b5b-bfc4-a187f7153a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:47.116248 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b32b3c-fb72-4b5b-bfc4-a187f7153a7c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:47.116947 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b32b3c-fb72-4b5b-bfc4-a187f7153a7c None admin] RBAC: Adding query filter params (interface=admin, service_id=c4d1fea9a64b4822951d9245af292a44, region_id=RegionOne) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:47.117256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b32b3c-fb72-4b5b-bfc4-a187f7153a7c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:47.118615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b32b3c-fb72-4b5b-bfc4-a187f7153a7c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:47.124914 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 492/995] 10.222.0.42 () {56 vars in 1249 bytes} [Wed Sep 14 00:52:46 2022] GET /identity/v3/endpoints?service_id=c4d1fea9a64b4822951d9245af292a44&interface=admin®ion_id=RegionOne => generated 193 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:47.129601 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c36e2-9bc4-4a7f-abc7-84df1b189cae None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:47.232109 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c36e2-9bc4-4a7f-abc7-84df1b189cae None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:47.267118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684e8ba-ed23-4aaa-b891-8e8538be46f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:47.267758 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684e8ba-ed23-4aaa-b891-8e8538be46f3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:47.268036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684e8ba-ed23-4aaa-b891-8e8538be46f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:47.268304 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684e8ba-ed23-4aaa-b891-8e8538be46f3 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:47.268902 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2684e8ba-ed23-4aaa-b891-8e8538be46f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:47.270260 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2684e8ba-ed23-4aaa-b891-8e8538be46f3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:47.283592 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 504/996] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Sep 14 00:52:47 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 00:52:47.418432 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4e255-fbca-4cec-a0d4-f1860998af3b None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:47.418997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4e255-fbca-4cec-a0d4-f1860998af3b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:47.418997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4e255-fbca-4cec-a0d4-f1860998af3b None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:47.420433 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 493/997] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:47.426387 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445fcf6-23e3-489c-bff8-90ecf20e0881 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:47.426682 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445fcf6-23e3-489c-bff8-90ecf20e0881 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:47.426956 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445fcf6-23e3-489c-bff8-90ecf20e0881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:47.446361 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-9445fcf6-23e3-489c-bff8-90ecf20e0881 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:47.447582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9445fcf6-23e3-489c-bff8-90ecf20e0881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:47.456413 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:47.525025 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 505/998] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:47 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:47.793638 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4e255-fbca-4cec-a0d4-f1860998af3b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:47.829330 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca71faa-e1c4-4f39-8a92-db7715b28f98 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:47.829861 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca71faa-e1c4-4f39-8a92-db7715b28f98 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:47.830143 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca71faa-e1c4-4f39-8a92-db7715b28f98 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:47.831472 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 506/999] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:52:47.836567 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97401adc-97a2-4566-82e3-95e6b9548ca1 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:47.836851 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97401adc-97a2-4566-82e3-95e6b9548ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:47.837125 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97401adc-97a2-4566-82e3-95e6b9548ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:47.855363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-97401adc-97a2-4566-82e3-95e6b9548ca1 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:47.856577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-97401adc-97a2-4566-82e3-95e6b9548ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:47.902940 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:47.910888 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4e255-fbca-4cec-a0d4-f1860998af3b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:47.925675 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 507/1000] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:47 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 00:52:47.945638 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e94f1-cf75-487e-999f-aba96bacde2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:47.946274 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e94f1-cf75-487e-999f-aba96bacde2e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:47.946558 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e94f1-cf75-487e-999f-aba96bacde2e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:47.946836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e94f1-cf75-487e-999f-aba96bacde2e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:47.947419 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20e94f1-cf75-487e-999f-aba96bacde2e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:47.948868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20e94f1-cf75-487e-999f-aba96bacde2e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:47.958247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-a20e94f1-cf75-487e-999f-aba96bacde2e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 00:52:47.958247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:47.958247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:52:47.958247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:52:47.958247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:52:47.958247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:52:47.958247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:52:47.958247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:47.958247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 00:52:47.958247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:47.958247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:47.958247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:47.958247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:47.958247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:47.958247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:47.958247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:47.958247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:47.959351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:47.959351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:47.959351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:47.959351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:47.959351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:47.959351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:47.959351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:52:47.959351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:52:47.959351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:52:47.959351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:52:47.959351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:47.959351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:47.959351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:47.959351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:47.959351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:47.959351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:47.960556 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:47.960556 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:47.960556 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:47.960556 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:47.960556 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:47.960556 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:47.960556 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:47.960556 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:47.960556 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:47.960556 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:47.960556 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:52:47.960556 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:47.960556 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 00:52:47.960556 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:47.964343 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 494/1001] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:47.965629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97401adc-97a2-4566-82e3-95e6b9548ca1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:48.039117 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:48.070636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97401adc-97a2-4566-82e3-95e6b9548ca1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:48.105563 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c943d-3900-4ae9-9652-aa8beb72ed03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:48.106175 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c943d-3900-4ae9-9652-aa8beb72ed03 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:48.106451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c943d-3900-4ae9-9652-aa8beb72ed03 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:48.106720 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c943d-3900-4ae9-9652-aa8beb72ed03 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:48.107342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7c943d-3900-4ae9-9652-aa8beb72ed03 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:48.107703 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7c943d-3900-4ae9-9652-aa8beb72ed03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:48.109272 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7c943d-3900-4ae9-9652-aa8beb72ed03 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:48.118934 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 508/1002] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 00:52:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:48.120168 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e94f1-cf75-487e-999f-aba96bacde2e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:48.201022 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c943d-3900-4ae9-9652-aa8beb72ed03 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:48.226053 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e94f1-cf75-487e-999f-aba96bacde2e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:48.261338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154e745-1492-41ac-aa87-ba93c9474b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:48.261943 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154e745-1492-41ac-aa87-ba93c9474b6e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:48.262222 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154e745-1492-41ac-aa87-ba93c9474b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:48.262500 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154e745-1492-41ac-aa87-ba93c9474b6e None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:48.263121 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154e745-1492-41ac-aa87-ba93c9474b6e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:48.263424 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154e745-1492-41ac-aa87-ba93c9474b6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:48.264879 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154e745-1492-41ac-aa87-ba93c9474b6e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:48.270148 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 495/1003] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 00:52:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:48.275468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154e745-1492-41ac-aa87-ba93c9474b6e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:48.302630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:48.307327 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c943d-3900-4ae9-9652-aa8beb72ed03 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:48.343405 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42fa65f-6fb4-4a32-b56f-46d09d83e575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:48.343999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fa65f-6fb4-4a32-b56f-46d09d83e575 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:48.344281 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fa65f-6fb4-4a32-b56f-46d09d83e575 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:48.344555 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fa65f-6fb4-4a32-b56f-46d09d83e575 None admin] PATH_INFO: `/v3/services/image` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:48.345134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42fa65f-6fb4-4a32-b56f-46d09d83e575 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:48.346506 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42fa65f-6fb4-4a32-b56f-46d09d83e575 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:48.349988 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:48.357466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-b42fa65f-6fb4-4a32-b56f-46d09d83e575 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 00:52:48.357466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:48.357466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:52:48.357466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:52:48.357466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:52:48.357466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:52:48.357466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:52:48.357466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:48.357466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 00:52:48.357466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:48.357466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:48.357466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:48.357466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:48.357466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:48.357466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:48.357466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:48.357466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:48.358751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:48.358751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:48.358751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:48.358751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:48.358751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:48.358751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:48.358751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:52:48.358751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:52:48.358751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:52:48.358751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:52:48.358751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:48.358751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:48.358751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:48.358751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:48.358751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:48.358751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:48.360103 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:48.360103 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:48.360103 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:48.360103 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:48.360103 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:48.360103 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:48.360103 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:48.360103 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:48.360103 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:48.360103 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:48.360103 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:52:48.360103 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:48.360103 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 00:52:48.360103 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:48.363830 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 509/1004] 10.222.0.42 () {56 vars in 1105 bytes} [Wed Sep 14 00:52:48 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:52:48.364676 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42fa65f-6fb4-4a32-b56f-46d09d83e575 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:48.382372 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154e745-1492-41ac-aa87-ba93c9474b6e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:48.416120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f7917-f1ed-4867-bab1-e1d8924fed5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:48.416838 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f7917-f1ed-4867-bab1-e1d8924fed5c None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:48.417114 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f7917-f1ed-4867-bab1-e1d8924fed5c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:48.417380 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f7917-f1ed-4867-bab1-e1d8924fed5c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:48.418029 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f7917-f1ed-4867-bab1-e1d8924fed5c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:48.419391 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f7917-f1ed-4867-bab1-e1d8924fed5c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:48.440632 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 496/1005] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 00:52:48 2022] POST /identity/v3/endpoints => generated 334 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 00:52:48.491810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42fa65f-6fb4-4a32-b56f-46d09d83e575 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:48.537820 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f695b1bb-e5b8-497d-b39c-24a825265c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:48.538443 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695b1bb-e5b8-497d-b39c-24a825265c4d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:48.538724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695b1bb-e5b8-497d-b39c-24a825265c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:48.539068 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695b1bb-e5b8-497d-b39c-24a825265c4d None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:48.539674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f695b1bb-e5b8-497d-b39c-24a825265c4d None admin] RBAC: Adding query filter params (name=image) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:48.539985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f695b1bb-e5b8-497d-b39c-24a825265c4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:48.541454 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f695b1bb-e5b8-497d-b39c-24a825265c4d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:48.552376 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 510/1006] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:48.553679 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f7917-f1ed-4867-bab1-e1d8924fed5c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:48.647266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:48.669493 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f7917-f1ed-4867-bab1-e1d8924fed5c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:48.706823 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35382d9-d4b9-48da-a52d-bbe9f47f5e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:48.707442 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35382d9-d4b9-48da-a52d-bbe9f47f5e2c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:48.707758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35382d9-d4b9-48da-a52d-bbe9f47f5e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:48.708031 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35382d9-d4b9-48da-a52d-bbe9f47f5e2c None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:48.708661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35382d9-d4b9-48da-a52d-bbe9f47f5e2c None admin] RBAC: Adding query filter params (type=image) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:48.708965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35382d9-d4b9-48da-a52d-bbe9f47f5e2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:48.710337 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35382d9-d4b9-48da-a52d-bbe9f47f5e2c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:48.714840 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 497/1007] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:48 2022] GET /identity/v3/services?type=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:49.507754 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef0d1c-6380-4116-b9d4-420fac8ab328 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:49.508247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef0d1c-6380-4116-b9d4-420fac8ab328 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:49.508345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef0d1c-6380-4116-b9d4-420fac8ab328 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:49.509802 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 511/1008] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:49.515648 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a79c4b-a84b-4603-aa0f-ded3be3dcab2 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:49.516046 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a79c4b-a84b-4603-aa0f-ded3be3dcab2 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:49.516231 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a79c4b-a84b-4603-aa0f-ded3be3dcab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:49.534426 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-10a79c4b-a84b-4603-aa0f-ded3be3dcab2 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:49.535643 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-10a79c4b-a84b-4603-aa0f-ded3be3dcab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:49.605767 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 498/1009] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:49 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:49.879736 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef0d1c-6380-4116-b9d4-420fac8ab328 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:49.940157 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:49.985364 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef0d1c-6380-4116-b9d4-420fac8ab328 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:50.018540 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a78127-5c98-40f5-9913-2a61ef37ca8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:50.019171 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a78127-5c98-40f5-9913-2a61ef37ca8a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:50.019449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a78127-5c98-40f5-9913-2a61ef37ca8a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:50.019748 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a78127-5c98-40f5-9913-2a61ef37ca8a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:50.020328 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a78127-5c98-40f5-9913-2a61ef37ca8a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:50.021683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a78127-5c98-40f5-9913-2a61ef37ca8a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:50.031001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-10a78127-5c98-40f5-9913-2a61ef37ca8a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 00:52:50.031001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:50.031001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:52:50.031001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:52:50.031001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:52:50.031001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:52:50.031001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:52:50.031001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:50.031001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 00:52:50.031001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:50.031001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:50.031001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:50.031001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:50.031001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:50.031001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:50.031001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:50.031001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:50.032202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:50.032202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:50.032202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:50.032202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:50.032202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:50.032202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:50.032202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:52:50.032202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:52:50.032202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:52:50.032202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:52:50.032202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:50.032202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:50.032202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:50.032202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:50.032202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:50.032202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:50.033070 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:50.033070 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:50.033070 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:50.033070 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:50.033070 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:50.033070 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:50.033070 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:50.033070 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:50.033070 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:50.033070 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:50.033070 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:52:50.033070 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:50.033070 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 00:52:50.033070 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:50.034317 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 512/1010] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Sep 14 00:52:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:50.037975 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a79c4b-a84b-4603-aa0f-ded3be3dcab2 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:50.121466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:50.147614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a79c4b-a84b-4603-aa0f-ded3be3dcab2 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:50.181960 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7da2c-9882-4b79-ab4d-0a945dc2a278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:50.182568 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7da2c-9882-4b79-ab4d-0a945dc2a278 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:50.182846 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7da2c-9882-4b79-ab4d-0a945dc2a278 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:50.183118 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7da2c-9882-4b79-ab4d-0a945dc2a278 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:50.183769 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7da2c-9882-4b79-ab4d-0a945dc2a278 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:50.184075 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7da2c-9882-4b79-ab4d-0a945dc2a278 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:50.185438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7da2c-9882-4b79-ab4d-0a945dc2a278 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:50.190833 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 499/1011] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Sep 14 00:52:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:50.196383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a78127-5c98-40f5-9913-2a61ef37ca8a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:50.297763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a78127-5c98-40f5-9913-2a61ef37ca8a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:50.321431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:50.334194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1535d-a5b2-48f1-b59a-8aa584aa23bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:50.334804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1535d-a5b2-48f1-b59a-8aa584aa23bf None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:50.335082 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1535d-a5b2-48f1-b59a-8aa584aa23bf None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:50.335352 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1535d-a5b2-48f1-b59a-8aa584aa23bf None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:50.336001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1535d-a5b2-48f1-b59a-8aa584aa23bf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:50.336301 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1535d-a5b2-48f1-b59a-8aa584aa23bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:50.337667 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1535d-a5b2-48f1-b59a-8aa584aa23bf None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:50.348057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 513/1012] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Sep 14 00:52:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:50.349285 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7da2c-9882-4b79-ab4d-0a945dc2a278 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:50.451246 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7da2c-9882-4b79-ab4d-0a945dc2a278 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:50.492976 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca5343-d9f5-4810-a85f-e34766f191e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:50.493591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca5343-d9f5-4810-a85f-e34766f191e0 None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:50.493873 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca5343-d9f5-4810-a85f-e34766f191e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:50.494147 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca5343-d9f5-4810-a85f-e34766f191e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:50.494826 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca5343-d9f5-4810-a85f-e34766f191e0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:50.496290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca5343-d9f5-4810-a85f-e34766f191e0 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:50.507936 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:50.517025 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 500/1013] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 00:52:50 2022] POST /identity/v3/endpoints => generated 325 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 00:52:50.783304 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60881f44-8a4f-4c82-a988-4c03d903c610 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:50.783845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60881f44-8a4f-4c82-a988-4c03d903c610 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:50.783845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60881f44-8a4f-4c82-a988-4c03d903c610 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:50.785355 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 514/1014] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:50.791191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338e8b3-2385-4fc5-90fc-d9af8862c868 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:50.791516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338e8b3-2385-4fc5-90fc-d9af8862c868 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:50.791801 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338e8b3-2385-4fc5-90fc-d9af8862c868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:50.810498 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-0338e8b3-2385-4fc5-90fc-d9af8862c868 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:50.811915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0338e8b3-2385-4fc5-90fc-d9af8862c868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:50.886138 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf3fb0-3a80-4c6f-bf81-ebbd91d55d0d None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:50.886423 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf3fb0-3a80-4c6f-bf81-ebbd91d55d0d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:50.886703 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf3fb0-3a80-4c6f-bf81-ebbd91d55d0d None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:50.888009 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 501/1015] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:50 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:50.891596 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 515/1016] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:52:50.894065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11132fe9-9368-4d6c-88d9-0eaac00de028 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:50.894362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11132fe9-9368-4d6c-88d9-0eaac00de028 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:50.894642 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11132fe9-9368-4d6c-88d9-0eaac00de028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:50.913491 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-11132fe9-9368-4d6c-88d9-0eaac00de028 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:50.914697 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-11132fe9-9368-4d6c-88d9-0eaac00de028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:50.983728 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 502/1017] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:50 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:51.171156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf3fb0-3a80-4c6f-bf81-ebbd91d55d0d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:51.252564 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11132fe9-9368-4d6c-88d9-0eaac00de028 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:51.275836 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf3fb0-3a80-4c6f-bf81-ebbd91d55d0d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:51.313239 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:51.314229 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766f012-e849-4a68-af82-56a63bcb9680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:51.315004 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766f012-e849-4a68-af82-56a63bcb9680 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:51.315178 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766f012-e849-4a68-af82-56a63bcb9680 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:51.315418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766f012-e849-4a68-af82-56a63bcb9680 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:51.326647 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7766f012-e849-4a68-af82-56a63bcb9680 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:52:51.332080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-7766f012-e849-4a68-af82-56a63bcb9680 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 00:52:51.332080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:51.332080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:51.332080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:51.332080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:51.332080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:51.332080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:51.332080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:51.332080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:51.332080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:51.332080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:51.332080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:51.332080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:52:51.332080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:52:51.332080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:52:51.332080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:52:51.333077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:52:51.333077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:52:51.333077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:52:51.333077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:52:51.333077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:51.333077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:51.333077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:51.333077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:51.333077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:51.333077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:51.333077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:51.333077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:51.333077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:51.333077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:51.333077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:51.333077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:51.333952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:51.333952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:51.333952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:51.333952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:51.333952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:52:51.333952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:52:51.333952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:52:51.333952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:52:51.333952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:52:51.333952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:52:51.333952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 00:52:51.333952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:51.338129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 516/1018] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Sep 14 00:52:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:51.338946 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766f012-e849-4a68-af82-56a63bcb9680 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:51.363760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11132fe9-9368-4d6c-88d9-0eaac00de028 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:51.391789 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:51.406088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2c35a-fcd5-4e49-879e-7f4254e8feef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:51.406713 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2c35a-fcd5-4e49-879e-7f4254e8feef None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:51.406996 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2c35a-fcd5-4e49-879e-7f4254e8feef None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:51.407446 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2c35a-fcd5-4e49-879e-7f4254e8feef None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:51.407955 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a2c35a-fcd5-4e49-879e-7f4254e8feef None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:51.409337 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a2c35a-fcd5-4e49-879e-7f4254e8feef None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:51.436299 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 503/1019] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 00:52:51 2022] POST /identity/v3/services => generated 242 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:52:51.455155 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766f012-e849-4a68-af82-56a63bcb9680 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:51.505125 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d28f8-afb0-4230-ae4e-652691345e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:51.505748 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d28f8-afb0-4230-ae4e-652691345e03 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:51.506030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d28f8-afb0-4230-ae4e-652691345e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:51.506301 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d28f8-afb0-4230-ae4e-652691345e03 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:51.507001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d28f8-afb0-4230-ae4e-652691345e03 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:51.507279 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d28f8-afb0-4230-ae4e-652691345e03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:51.508715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9d28f8-afb0-4230-ae4e-652691345e03 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:51.523332 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 517/1020] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Sep 14 00:52:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:51.524996 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2c35a-fcd5-4e49-879e-7f4254e8feef None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:51.637076 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2c35a-fcd5-4e49-879e-7f4254e8feef None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:51.652085 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:51.674898 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133ff5c-4f4d-4adf-9db3-1d8fa94f74d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:51.675535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133ff5c-4f4d-4adf-9db3-1d8fa94f74d8 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:51.675855 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133ff5c-4f4d-4adf-9db3-1d8fa94f74d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:51.676136 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133ff5c-4f4d-4adf-9db3-1d8fa94f74d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:51.688026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133ff5c-4f4d-4adf-9db3-1d8fa94f74d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:51.689427 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133ff5c-4f4d-4adf-9db3-1d8fa94f74d8 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:51.704506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-9133ff5c-4f4d-4adf-9db3-1d8fa94f74d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:51.704506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:51.704506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:51.704506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:51.704506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:51.704506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:51.704506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:51.704506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:51.704506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:51.704506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:51.704506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:51.704506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:51.704506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:51.704506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:51.704506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:51.704506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:51.705636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:51.705636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:51.705636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:51.705636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:51.705636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:51.705636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:51.705636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:51.705636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:51.705636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:51.705636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:51.705636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:51.705636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:51.705636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:51.705636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:51.705636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:51.705636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:51.705636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:51.706675 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:51.706675 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:51.706675 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:51.706675 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 504/1021] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:51.711381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d28f8-afb0-4230-ae4e-652691345e03 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:51.812451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d28f8-afb0-4230-ae4e-652691345e03 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:51.819480 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:51.848593 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdded66-fbb2-423d-834b-6e728d28e319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:51.849206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdded66-fbb2-423d-834b-6e728d28e319 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:51.849484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdded66-fbb2-423d-834b-6e728d28e319 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:51.849757 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdded66-fbb2-423d-834b-6e728d28e319 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:51.850390 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdded66-fbb2-423d-834b-6e728d28e319 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:51.850689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdded66-fbb2-423d-834b-6e728d28e319 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:51.852085 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdded66-fbb2-423d-834b-6e728d28e319 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:51.864172 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 518/1022] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:51.869551 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133ff5c-4f4d-4adf-9db3-1d8fa94f74d8 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:51.971346 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133ff5c-4f4d-4adf-9db3-1d8fa94f74d8 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:52.005481 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21bb890-7772-4e5e-a5be-a7f02f818132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:52.006111 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bb890-7772-4e5e-a5be-a7f02f818132 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:52.006395 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bb890-7772-4e5e-a5be-a7f02f818132 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:52.006671 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21bb890-7772-4e5e-a5be-a7f02f818132 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:52.014819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21bb890-7772-4e5e-a5be-a7f02f818132 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:52.016597 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21bb890-7772-4e5e-a5be-a7f02f818132 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:52.027760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-e21bb890-7772-4e5e-a5be-a7f02f818132 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 00:52:52.027760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:52.027760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:52.027760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:52.027760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:52.027760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:52.027760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:52.027760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:52.027760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:52.027760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:52.027760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:52.027760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:52.027760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:52:52.027760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:52:52.027760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:52:52.027760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:52:52.028701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:52.028701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:52.028701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:52:52.028701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:52.028701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:52:52.028701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:52.028701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:52.028701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:52.028701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:52.028701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:52.028701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:52.028701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:52.028701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:52.028701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:52.028701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:52.028701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:52.028701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:52.028701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:52.030500 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:52.030500 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:52.030500 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:52:52.030500 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:52:52.030500 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:52:52.030500 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:52:52.030500 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:52:52.030500 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:52:52.030500 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 00:52:52.030500 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:52.031776 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 505/1023] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Sep 14 00:52:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:52.035608 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdded66-fbb2-423d-834b-6e728d28e319 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:52.137668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdded66-fbb2-423d-834b-6e728d28e319 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:52.174948 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2cd8dc-b9b7-4a7c-ab09-9f796405aea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:52.175589 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cd8dc-b9b7-4a7c-ab09-9f796405aea3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:52.175880 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cd8dc-b9b7-4a7c-ab09-9f796405aea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:52.176161 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cd8dc-b9b7-4a7c-ab09-9f796405aea3 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:52.176898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2cd8dc-b9b7-4a7c-ab09-9f796405aea3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:52.177209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2cd8dc-b9b7-4a7c-ab09-9f796405aea3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:52.178652 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2cd8dc-b9b7-4a7c-ab09-9f796405aea3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:52.202090 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 519/1024] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Sep 14 00:52:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:52.203194 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21bb890-7772-4e5e-a5be-a7f02f818132 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:52.231760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:52.306761 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21bb890-7772-4e5e-a5be-a7f02f818132 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:52.340673 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8cef08-91a9-44fb-a756-9ad40b88f870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:52.341435 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8cef08-91a9-44fb-a756-9ad40b88f870 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:52.341820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8cef08-91a9-44fb-a756-9ad40b88f870 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:52.342738 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8cef08-91a9-44fb-a756-9ad40b88f870 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:52.413078 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8cef08-91a9-44fb-a756-9ad40b88f870 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:52.414684 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8cef08-91a9-44fb-a756-9ad40b88f870 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:52.440280 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:52.520434 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 506/1025] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:52:52 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 00:52:52.526285 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2cd8dc-b9b7-4a7c-ab09-9f796405aea3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:52.552055 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:52.631019 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2cd8dc-b9b7-4a7c-ab09-9f796405aea3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:52.667932 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245bd367-b7a5-437f-bff8-c30943cbb119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:52.668566 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bd367-b7a5-437f-bff8-c30943cbb119 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:52.668846 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bd367-b7a5-437f-bff8-c30943cbb119 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:52.669119 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bd367-b7a5-437f-bff8-c30943cbb119 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:52.679377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245bd367-b7a5-437f-bff8-c30943cbb119 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:52.680847 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245bd367-b7a5-437f-bff8-c30943cbb119 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:52.695745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-245bd367-b7a5-437f-bff8-c30943cbb119 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:52.695745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:52.695745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:52.695745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:52.695745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:52.695745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:52.695745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:52.695745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:52.695745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:52.695745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:52.695745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:52.695745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:52.695745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:52.695745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:52.695745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:52.695745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:52.696723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:52.696723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:52.696723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:52.696723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:52.696723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:52.696723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:52.696723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:52.696723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:52.696723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:52.696723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:52.696723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:52.696723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:52.696723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:52.696723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:52.696723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:52.696723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:52.696723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:52.702883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:52.702883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:52.702883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:52.702883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 520/1026] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:52.703114 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8cef08-91a9-44fb-a756-9ad40b88f870 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:52.739045 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384aabd-f9c9-4e12-a228-7d1baf5e4471 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:52.739387 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384aabd-f9c9-4e12-a228-7d1baf5e4471 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:52.739630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384aabd-f9c9-4e12-a228-7d1baf5e4471 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:52.741099 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 521/1027] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:52:52.745996 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea226fa-75d9-4277-a7f7-40329c343890 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:52.746276 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea226fa-75d9-4277-a7f7-40329c343890 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:52.746549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea226fa-75d9-4277-a7f7-40329c343890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:52.765150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-cea226fa-75d9-4277-a7f7-40329c343890 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:52.766891 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cea226fa-75d9-4277-a7f7-40329c343890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:52.807011 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8cef08-91a9-44fb-a756-9ad40b88f870 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:52.833816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:52.839885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 522/1028] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:52.844054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b902008-0dcf-4648-ac9f-16805941a1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:52.844872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b902008-0dcf-4648-ac9f-16805941a1a1 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:52.845394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b902008-0dcf-4648-ac9f-16805941a1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:52.845849 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b902008-0dcf-4648-ac9f-16805941a1a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:52.846677 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b902008-0dcf-4648-ac9f-16805941a1a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:52.847153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b902008-0dcf-4648-ac9f-16805941a1a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:52.848722 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b902008-0dcf-4648-ac9f-16805941a1a1 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:52.862431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 507/1029] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:52.867698 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea226fa-75d9-4277-a7f7-40329c343890 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:52.877504 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:52.971452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea226fa-75d9-4277-a7f7-40329c343890 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:53.005768 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c432e67-43b4-4f46-9674-dcfe09ed8a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:53.006393 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c432e67-43b4-4f46-9674-dcfe09ed8a05 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:53.006664 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c432e67-43b4-4f46-9674-dcfe09ed8a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:53.006933 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c432e67-43b4-4f46-9674-dcfe09ed8a05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:53.013568 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c432e67-43b4-4f46-9674-dcfe09ed8a05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:53.015953 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c432e67-43b4-4f46-9674-dcfe09ed8a05 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:53.026947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-3c432e67-43b4-4f46-9674-dcfe09ed8a05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:53.026947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:53.026947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:53.026947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:53.026947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:53.026947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:53.026947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:53.026947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:53.026947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:53.026947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:53.026947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:53.026947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:53.026947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:52:53.026947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:52:53.026947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:52:53.028094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:52:53.028094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:53.028094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:53.028094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:53.028094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:53.028094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:53.028094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:53.028094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:53.028094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:53.028094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:53.028094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:53.028094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:53.028094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:53.028094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:53.028094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:53.028094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:53.028094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:53.029386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:52:53.029386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:52:53.029386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:52:53.029386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:52:53.029386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:52:53.029386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:52:53.029386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:53.029386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:53.031892 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 523/1030] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:53.034083 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b902008-0dcf-4648-ac9f-16805941a1a1 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:53.106756 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c432e67-43b4-4f46-9674-dcfe09ed8a05 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:53.142719 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b902008-0dcf-4648-ac9f-16805941a1a1 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:53.177951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1035e98-2281-4370-80eb-e679c8aae586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:53.178560 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1035e98-2281-4370-80eb-e679c8aae586 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:53.178840 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1035e98-2281-4370-80eb-e679c8aae586 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:53.179114 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1035e98-2281-4370-80eb-e679c8aae586 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:53.179836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1035e98-2281-4370-80eb-e679c8aae586 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:53.180140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1035e98-2281-4370-80eb-e679c8aae586 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:53.180336 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:53.181576 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1035e98-2281-4370-80eb-e679c8aae586 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:53.199758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 508/1031] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:52:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:53.201012 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1035e98-2281-4370-80eb-e679c8aae586 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:53.223631 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c432e67-43b4-4f46-9674-dcfe09ed8a05 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:53.257106 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:53.263055 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b26800-9b5a-4ee2-90dd-15d8551e93fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:53.264194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b26800-9b5a-4ee2-90dd-15d8551e93fc None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:53.264480 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b26800-9b5a-4ee2-90dd-15d8551e93fc None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:53.264755 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b26800-9b5a-4ee2-90dd-15d8551e93fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:53.266249 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b26800-9b5a-4ee2-90dd-15d8551e93fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:53.268257 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b26800-9b5a-4ee2-90dd-15d8551e93fc None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:53.270142 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 524/1032] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:53.275673 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b26800-9b5a-4ee2-90dd-15d8551e93fc None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:53.313821 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1035e98-2281-4370-80eb-e679c8aae586 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:53.350140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd8158-c2b6-4bf3-b3dd-980c26899df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:53.350820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd8158-c2b6-4bf3-b3dd-980c26899df5 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:53.351101 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd8158-c2b6-4bf3-b3dd-980c26899df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:53.351377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd8158-c2b6-4bf3-b3dd-980c26899df5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:53.352153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd8158-c2b6-4bf3-b3dd-980c26899df5 None admin] RBAC: Adding query filter params (role.id=ae888494799042399af96f094898c4ed, scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3, user.id=5e0fe2e84f6748c992c9beedf2f5d49b) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:53.352464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd8158-c2b6-4bf3-b3dd-980c26899df5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:53.353895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd8158-c2b6-4bf3-b3dd-980c26899df5 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:53.360628 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 509/1033] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:52:53 2022] GET /identity/v3/role_assignments?user.id=5e0fe2e84f6748c992c9beedf2f5d49b&scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3&role.id=ae888494799042399af96f094898c4ed => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:53.382806 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b26800-9b5a-4ee2-90dd-15d8551e93fc None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:53.422551 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36f18e-2cf2-48e1-87f0-dfafd70cfbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:53.423187 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36f18e-2cf2-48e1-87f0-dfafd70cfbd7 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:53.423464 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36f18e-2cf2-48e1-87f0-dfafd70cfbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:53.423763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36f18e-2cf2-48e1-87f0-dfafd70cfbd7 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:53.424435 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36f18e-2cf2-48e1-87f0-dfafd70cfbd7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:53.425885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36f18e-2cf2-48e1-87f0-dfafd70cfbd7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:53.431312 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:53.458503 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 525/1034] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 00:52:53 2022] POST /identity/v3/projects => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 00:52:53.663114 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcacde1-6b42-4e2a-a0dc-7c0fc49d940c None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:53.663405 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcacde1-6b42-4e2a-a0dc-7c0fc49d940c None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:53.663722 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcacde1-6b42-4e2a-a0dc-7c0fc49d940c None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:53.665087 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 510/1035] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:53.671232 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c1ed8-5445-4d01-9afd-bf0c456b9dcd None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:53.671736 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c1ed8-5445-4d01-9afd-bf0c456b9dcd None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:53.672022 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c1ed8-5445-4d01-9afd-bf0c456b9dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:53.690515 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d52c1ed8-5445-4d01-9afd-bf0c456b9dcd None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:53.691749 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d52c1ed8-5445-4d01-9afd-bf0c456b9dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:53.758406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 526/1036] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:54.027366 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcacde1-6b42-4e2a-a0dc-7c0fc49d940c None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:54.134000 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcacde1-6b42-4e2a-a0dc-7c0fc49d940c None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:54.145289 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:54.171382 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de539c8e-9af1-4b59-83f7-c22a19da9028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:54.172055 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de539c8e-9af1-4b59-83f7-c22a19da9028 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:54.172339 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de539c8e-9af1-4b59-83f7-c22a19da9028 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:54.172614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de539c8e-9af1-4b59-83f7-c22a19da9028 None admin] PATH_INFO: `/v3/services/image` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:54.173206 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de539c8e-9af1-4b59-83f7-c22a19da9028 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:54.174579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de539c8e-9af1-4b59-83f7-c22a19da9028 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:54.183524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-de539c8e-9af1-4b59-83f7-c22a19da9028 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 00:52:54.183524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:54.183524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:52:54.183524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:52:54.183524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:52:54.183524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:52:54.183524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:52:54.183524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:54.183524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 00:52:54.183524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:54.183524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:54.183524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:54.183524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:54.183524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:54.183524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:54.183524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:54.183524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:54.184542 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:54.184542 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:54.184542 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:54.184542 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:54.184542 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:54.184542 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:54.184542 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:52:54.184542 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:52:54.184542 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:52:54.184542 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:52:54.184542 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:54.184542 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:54.184542 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:54.184542 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:54.184542 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:54.184542 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:54.185404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:54.185404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:54.185404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:54.185404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:54.185404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:54.185404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:54.185404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:54.185404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:54.185404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:54.185404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:54.185404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:52:54.185404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:54.185404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 00:52:54.185404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:54.187889 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 511/1037] 10.222.0.42 () {56 vars in 1105 bytes} [Wed Sep 14 00:52:54 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:54.190629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c1ed8-5445-4d01-9afd-bf0c456b9dcd None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:54.296875 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c1ed8-5445-4d01-9afd-bf0c456b9dcd None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:54.313303 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:54.333534 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfcca5a-1c3a-4d7f-9ba9-0f8beb00b92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:54.334145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcca5a-1c3a-4d7f-9ba9-0f8beb00b92d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:54.334423 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcca5a-1c3a-4d7f-9ba9-0f8beb00b92d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:54.334737 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcca5a-1c3a-4d7f-9ba9-0f8beb00b92d None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:54.335339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfcca5a-1c3a-4d7f-9ba9-0f8beb00b92d None admin] RBAC: Adding query filter params (name=image) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:54.335664 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfcca5a-1c3a-4d7f-9ba9-0f8beb00b92d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:54.337025 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfcca5a-1c3a-4d7f-9ba9-0f8beb00b92d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:54.347335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 527/1038] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:54.348303 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de539c8e-9af1-4b59-83f7-c22a19da9028 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:54.455315 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de539c8e-9af1-4b59-83f7-c22a19da9028 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:54.497976 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32663b71-ede1-49e8-9fb9-3c678fa4f21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:54.498601 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32663b71-ede1-49e8-9fb9-3c678fa4f21c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:54.498878 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32663b71-ede1-49e8-9fb9-3c678fa4f21c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:54.499157 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32663b71-ede1-49e8-9fb9-3c678fa4f21c None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:54.499809 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32663b71-ede1-49e8-9fb9-3c678fa4f21c None admin] RBAC: Adding query filter params (type=image) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:54.500119 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32663b71-ede1-49e8-9fb9-3c678fa4f21c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:54.501491 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32663b71-ede1-49e8-9fb9-3c678fa4f21c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:54.511013 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 512/1039] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:54.512307 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfcca5a-1c3a-4d7f-9ba9-0f8beb00b92d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:54.616743 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfcca5a-1c3a-4d7f-9ba9-0f8beb00b92d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:54.653295 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5af50-1d33-471f-955e-6c37208155ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:54.653295 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5af50-1d33-471f-955e-6c37208155ec None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:54.653295 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5af50-1d33-471f-955e-6c37208155ec None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:54.653295 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5af50-1d33-471f-955e-6c37208155ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:54.655342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5af50-1d33-471f-955e-6c37208155ec None admin] RBAC: Adding query filter params (interface=public, service_id=970baba309ec4a04b77e299c542e9cab, region_id=RegionOne) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:54.655342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5af50-1d33-471f-955e-6c37208155ec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:54.655845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5af50-1d33-471f-955e-6c37208155ec None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:54.661262 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 528/1040] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Sep 14 00:52:54 2022] GET /identity/v3/endpoints?service_id=970baba309ec4a04b77e299c542e9cab&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:54.666913 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32663b71-ede1-49e8-9fb9-3c678fa4f21c None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:54.709435 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:54.773274 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32663b71-ede1-49e8-9fb9-3c678fa4f21c None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:54.808923 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b84e8-3d65-4201-9644-0f623d41da61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:54.809759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b84e8-3d65-4201-9644-0f623d41da61 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:54.810189 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b84e8-3d65-4201-9644-0f623d41da61 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:54.811138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b84e8-3d65-4201-9644-0f623d41da61 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:54.812072 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b84e8-3d65-4201-9644-0f623d41da61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:54.813599 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b84e8-3d65-4201-9644-0f623d41da61 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:54.820783 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 513/1041] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Sep 14 00:52:54 2022] GET /identity/v3/services => generated 1957 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 00:52:55.539538 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca275c6-ae7b-4f98-9cc7-b5325e247a00 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:55.540101 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca275c6-ae7b-4f98-9cc7-b5325e247a00 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:55.540101 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca275c6-ae7b-4f98-9cc7-b5325e247a00 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:55.541616 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 529/1042] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:55.547653 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21352ad-9c13-4122-a7ca-507cc8adf8ff None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:55.548091 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21352ad-9c13-4122-a7ca-507cc8adf8ff None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:55.548275 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21352ad-9c13-4122-a7ca-507cc8adf8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:55.568333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d21352ad-9c13-4122-a7ca-507cc8adf8ff None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:55.569579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d21352ad-9c13-4122-a7ca-507cc8adf8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:55.642747 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 514/1043] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:55.678538 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b776be-00b7-4817-bd00-6e0c1fc092d1 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:55.678862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b776be-00b7-4817-bd00-6e0c1fc092d1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:55.679095 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b776be-00b7-4817-bd00-6e0c1fc092d1 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:55.680587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 530/1044] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:55.686160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fc8a9-895d-41d4-8240-9e0875367eea None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:55.686503 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fc8a9-895d-41d4-8240-9e0875367eea None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:55.686766 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fc8a9-895d-41d4-8240-9e0875367eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:55.706870 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-303fc8a9-895d-41d4-8240-9e0875367eea None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:55.708089 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-303fc8a9-895d-41d4-8240-9e0875367eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:55.734851 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:55.776985 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 515/1045] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:55.908663 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b776be-00b7-4817-bd00-6e0c1fc092d1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:55.956493 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:56.011965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b776be-00b7-4817-bd00-6e0c1fc092d1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:56.047338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303fc8a9-895d-41d4-8240-9e0875367eea None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:56.047707 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32724ff-f531-4796-9efa-3545419a3833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:56.048352 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32724ff-f531-4796-9efa-3545419a3833 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:56.048630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32724ff-f531-4796-9efa-3545419a3833 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:56.048901 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32724ff-f531-4796-9efa-3545419a3833 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:56.061690 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32724ff-f531-4796-9efa-3545419a3833 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:52:56.067030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-d32724ff-f531-4796-9efa-3545419a3833 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 00:52:56.067030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:56.067030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:56.067030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:56.067030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:56.067030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:56.067030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:56.067030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:56.067030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:56.067030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:56.067030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:56.067030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:56.067030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:52:56.067030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:52:56.067030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:52:56.067030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:52:56.068225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:52:56.068225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:52:56.068225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:52:56.068225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:52:56.068225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:56.068225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:56.068225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:56.068225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:56.068225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:56.068225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:56.068225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:56.068225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:56.068225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:56.068225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:56.068225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:56.068225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:56.069539 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:56.069539 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:56.069539 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:56.069539 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:56.069539 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:52:56.069539 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:52:56.069539 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:52:56.069539 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:52:56.069539 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:52:56.069539 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:52:56.069539 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 00:52:56.069539 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:56.069539 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 531/1046] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Sep 14 00:52:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:56.073965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32724ff-f531-4796-9efa-3545419a3833 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:56.169558 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303fc8a9-895d-41d4-8240-9e0875367eea None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:56.181727 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32724ff-f531-4796-9efa-3545419a3833 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:56.184804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:56.219335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788eb69-d9d3-48a9-a004-62060ac6d42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:56.219998 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788eb69-d9d3-48a9-a004-62060ac6d42c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:56.220283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788eb69-d9d3-48a9-a004-62060ac6d42c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:56.220558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788eb69-d9d3-48a9-a004-62060ac6d42c None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:56.221225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788eb69-d9d3-48a9-a004-62060ac6d42c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:56.221533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788eb69-d9d3-48a9-a004-62060ac6d42c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:56.221749 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9020e-bab1-4b7b-a5bc-03a3be42bbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:56.222390 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9020e-bab1-4b7b-a5bc-03a3be42bbed None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:56.222667 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9020e-bab1-4b7b-a5bc-03a3be42bbed None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:56.222925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788eb69-d9d3-48a9-a004-62060ac6d42c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:56.223033 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9020e-bab1-4b7b-a5bc-03a3be42bbed None admin] PATH_INFO: `/v3/domains/default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:56.224066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9020e-bab1-4b7b-a5bc-03a3be42bbed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:56.225490 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9020e-bab1-4b7b-a5bc-03a3be42bbed None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:56.231645 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 516/1047] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:56.231780 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 532/1048] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Sep 14 00:52:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:56.236489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9020e-bab1-4b7b-a5bc-03a3be42bbed None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:56.237647 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788eb69-d9d3-48a9-a004-62060ac6d42c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:56.268061 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:56.342367 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9020e-bab1-4b7b-a5bc-03a3be42bbed None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:56.345164 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788eb69-d9d3-48a9-a004-62060ac6d42c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:56.378368 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da7050-2200-431d-998d-9baca62ddb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:56.378985 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da7050-2200-431d-998d-9baca62ddb82 None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:56.379265 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da7050-2200-431d-998d-9baca62ddb82 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:56.379541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da7050-2200-431d-998d-9baca62ddb82 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:56.380217 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da7050-2200-431d-998d-9baca62ddb82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:56.380722 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1b044-91a0-40df-b094-306f0e52f5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:56.381347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1b044-91a0-40df-b094-306f0e52f5e5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:56.381626 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1b044-91a0-40df-b094-306f0e52f5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:56.381735 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da7050-2200-431d-998d-9baca62ddb82 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:56.381899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1b044-91a0-40df-b094-306f0e52f5e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:56.396597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec1b044-91a0-40df-b094-306f0e52f5e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:56.399209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec1b044-91a0-40df-b094-306f0e52f5e5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:56.408115 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 517/1049] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 00:52:56 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:52:56.421607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-8ec1b044-91a0-40df-b094-306f0e52f5e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:56.421607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:56.421607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:56.421607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:56.421607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:56.421607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:56.421607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:56.421607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:56.421607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:56.421607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:56.421607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:56.421607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:56.421607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:56.421607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:56.421607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:56.421607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:56.422857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:56.422857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:56.422857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:56.422857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:56.422857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:56.422857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:56.422857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:56.422857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:56.422857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:56.422857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:56.422857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:56.422857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:56.422857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:56.422857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:56.422857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:56.422857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:56.422857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:56.423986 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:56.423986 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:56.423986 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:56.425412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 533/1050] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:56.431284 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da7050-2200-431d-998d-9baca62ddb82 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:56.537185 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:56.556772 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da7050-2200-431d-998d-9baca62ddb82 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:56.595469 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561beae-6244-4580-9430-1706a7de3fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:56.596216 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561beae-6244-4580-9430-1706a7de3fbb None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:56.596509 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561beae-6244-4580-9430-1706a7de3fbb None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:56.596787 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561beae-6244-4580-9430-1706a7de3fbb None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:56.597437 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a561beae-6244-4580-9430-1706a7de3fbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:56.597746 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a561beae-6244-4580-9430-1706a7de3fbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:56.599122 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a561beae-6244-4580-9430-1706a7de3fbb None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:56.611769 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 518/1051] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:56.617577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1b044-91a0-40df-b094-306f0e52f5e5 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:56.638114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:56.725290 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1b044-91a0-40df-b094-306f0e52f5e5 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:56.760115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a4097-5fae-4f9d-9179-096b50c26d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:56.760749 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a4097-5fae-4f9d-9179-096b50c26d26 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:56.761031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a4097-5fae-4f9d-9179-096b50c26d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:56.761305 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a4097-5fae-4f9d-9179-096b50c26d26 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:56.769350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a4097-5fae-4f9d-9179-096b50c26d26 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:56.770796 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a4097-5fae-4f9d-9179-096b50c26d26 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:56.782517 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-ed6a4097-5fae-4f9d-9179-096b50c26d26 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 00:52:56.782517 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:56.782517 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:56.782517 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:56.782517 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:56.782517 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:56.782517 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:56.782517 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:56.782517 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:56.782517 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:56.782517 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:56.782517 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:56.782517 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:52:56.782517 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:52:56.782517 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:52:56.782517 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:52:56.783452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:56.783452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:56.783452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:52:56.783452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:56.783452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:52:56.783452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:52:56.783452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:56.783452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:56.783452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:56.783452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:56.783452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:56.783452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:56.783452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:56.783452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:56.783452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:56.783452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:56.783452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:56.783452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:56.785005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:56.785005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:56.785005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:52:56.785005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:52:56.785005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:52:56.785005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:52:56.785005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:52:56.785005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:52:56.785005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 00:52:56.785005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:56.785005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 534/1052] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Sep 14 00:52:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:52:56.789716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561beae-6244-4580-9430-1706a7de3fbb None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:56.894113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561beae-6244-4580-9430-1706a7de3fbb None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:56.932975 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30d9e0-cd2f-4d97-85a5-2b032dcec738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:56.933609 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30d9e0-cd2f-4d97-85a5-2b032dcec738 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:56.933892 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30d9e0-cd2f-4d97-85a5-2b032dcec738 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:56.934167 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30d9e0-cd2f-4d97-85a5-2b032dcec738 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:56.934911 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30d9e0-cd2f-4d97-85a5-2b032dcec738 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:56.935223 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30d9e0-cd2f-4d97-85a5-2b032dcec738 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:56.936760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30d9e0-cd2f-4d97-85a5-2b032dcec738 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:56.960193 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 519/1053] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Sep 14 00:52:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:56.961590 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a4097-5fae-4f9d-9179-096b50c26d26 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:57.027729 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f8d6f-3570-4ea7-b950-82a5e07517e6 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:57.028545 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f8d6f-3570-4ea7-b950-82a5e07517e6 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:57.028988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f8d6f-3570-4ea7-b950-82a5e07517e6 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:57.030824 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 520/1054] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:52:57.036029 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbe899-6526-4582-9771-9d9c5832061c None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:57.037223 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbe899-6526-4582-9771-9d9c5832061c None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:57.037641 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbe899-6526-4582-9771-9d9c5832061c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:57.046384 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:57.048962 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:57.065189 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-4bbbe899-6526-4582-9771-9d9c5832061c None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:57.067124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4bbbe899-6526-4582-9771-9d9c5832061c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:57.072200 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a4097-5fae-4f9d-9179-096b50c26d26 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:57.112109 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c86cd-4385-4123-926b-50c24f35064e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:57.112884 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c86cd-4385-4123-926b-50c24f35064e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:57.113379 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c86cd-4385-4123-926b-50c24f35064e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:57.113744 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c86cd-4385-4123-926b-50c24f35064e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:57.125345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c86cd-4385-4123-926b-50c24f35064e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:57.127020 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c86cd-4385-4123-926b-50c24f35064e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:57.137969 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 521/1055] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 00:52:57.142983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-d42c86cd-4385-4123-926b-50c24f35064e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:57.142983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:57.142983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:57.142983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:57.142983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:57.142983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:57.142983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:57.142983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:57.142983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:57.142983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:57.142983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:57.142983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:57.142983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:52:57.142983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:52:57.142983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:52:57.142983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:52:57.143965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:57.143965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:57.143965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:57.143965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:57.143965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:57.143965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:57.143965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:57.143965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:57.143965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:57.143965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:57.143965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:57.143965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:57.143965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:57.143965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:57.143965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:57.143965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:57.143965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:52:57.149882 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:52:57.149882 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:52:57.149882 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:57.149882 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 535/1056] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:57.150185 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbbe899-6526-4582-9771-9d9c5832061c None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:57.251459 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbbe899-6526-4582-9771-9d9c5832061c None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:57.287893 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81aeab5-2ea5-426c-8110-57f8c91879ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:57.288584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81aeab5-2ea5-426c-8110-57f8c91879ff None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:57.288930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81aeab5-2ea5-426c-8110-57f8c91879ff None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:57.289268 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81aeab5-2ea5-426c-8110-57f8c91879ff None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:57.289968 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81aeab5-2ea5-426c-8110-57f8c91879ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:57.290334 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81aeab5-2ea5-426c-8110-57f8c91879ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:57.291781 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81aeab5-2ea5-426c-8110-57f8c91879ff None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:57.293351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:57.309897 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 522/1057] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:52:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:57.312139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c86cd-4385-4123-926b-50c24f35064e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:57.409940 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81aeab5-2ea5-426c-8110-57f8c91879ff None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:57.419146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c86cd-4385-4123-926b-50c24f35064e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:57.448611 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:57.456350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01809c-2668-4960-8c20-48878fd11822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:57.457059 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01809c-2668-4960-8c20-48878fd11822 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:57.457406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01809c-2668-4960-8c20-48878fd11822 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:57.457744 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01809c-2668-4960-8c20-48878fd11822 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:57.464588 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff01809c-2668-4960-8c20-48878fd11822 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:57.466088 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff01809c-2668-4960-8c20-48878fd11822 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:57.481864 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-ff01809c-2668-4960-8c20-48878fd11822 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:57.481864 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:57.481864 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:57.481864 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:57.481864 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:57.481864 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:57.481864 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:57.481864 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:57.481864 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:57.481864 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:57.481864 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:57.481864 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:57.481864 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:52:57.481864 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:52:57.481864 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:52:57.482914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:52:57.482914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:57.482914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:57.482914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:57.482914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:57.482914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:57.482914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:57.482914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:57.482914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:57.482914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:57.482914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:57.482914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:57.482914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:57.482914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:57.482914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:57.482914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:57.482914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:57.483919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:52:57.483919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:52:57.483919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:52:57.483919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:52:57.483919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:52:57.483919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:52:57.483919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:52:57.483919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:52:57.483919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 536/1058] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:52:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:57.489478 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01809c-2668-4960-8c20-48878fd11822 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:57.530255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81aeab5-2ea5-426c-8110-57f8c91879ff None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:57.564008 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da753c24-6e0a-4eb2-a192-03c079a5cc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:57.564613 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da753c24-6e0a-4eb2-a192-03c079a5cc56 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:57.564846 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da753c24-6e0a-4eb2-a192-03c079a5cc56 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:57.565121 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da753c24-6e0a-4eb2-a192-03c079a5cc56 None admin] PATH_INFO: `/v3/services/image` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:57.565701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da753c24-6e0a-4eb2-a192-03c079a5cc56 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:57.567071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da753c24-6e0a-4eb2-a192-03c079a5cc56 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:57.575760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-da753c24-6e0a-4eb2-a192-03c079a5cc56 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 00:52:57.575760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:57.575760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:52:57.575760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:52:57.575760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:52:57.575760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:52:57.575760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:52:57.575760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:57.575760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 00:52:57.575760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:57.575760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:52:57.575760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:57.575760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:57.575760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:57.575760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:57.575760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:57.575760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:57.577310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:57.577310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:57.577310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:57.577310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:57.577310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:57.577310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:57.577310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:52:57.577310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:52:57.577310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:52:57.577310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:52:57.577310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:57.577310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:57.577310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:57.577310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:57.577310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:57.577310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:57.578327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:57.578327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:57.578327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:57.578327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:57.578327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:57.578327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:57.578327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:57.578327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:57.578327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:57.578327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:57.578327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:52:57.578327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:52:57.578327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 00:52:57.578327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:57.582743 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 523/1059] 10.222.0.42 () {56 vars in 1105 bytes} [Wed Sep 14 00:52:57 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:52:57.583464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da753c24-6e0a-4eb2-a192-03c079a5cc56 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:57.600540 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01809c-2668-4960-8c20-48878fd11822 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:57.641490 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b3555d-d86c-4ac8-9d4c-e862075681c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:57.642119 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3555d-d86c-4ac8-9d4c-e862075681c3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:57.642355 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3555d-d86c-4ac8-9d4c-e862075681c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:57.642630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3555d-d86c-4ac8-9d4c-e862075681c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:57.643314 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b3555d-d86c-4ac8-9d4c-e862075681c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:57.643646 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b3555d-d86c-4ac8-9d4c-e862075681c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:57.645069 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b3555d-d86c-4ac8-9d4c-e862075681c3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:57.648348 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:57.658593 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 537/1060] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:52:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:57.664339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b3555d-d86c-4ac8-9d4c-e862075681c3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:57.692437 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da753c24-6e0a-4eb2-a192-03c079a5cc56 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:57.724984 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:57.728752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2e92c-1a36-4844-88f7-21de7bf148a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:57.729357 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2e92c-1a36-4844-88f7-21de7bf148a0 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:57.729640 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2e92c-1a36-4844-88f7-21de7bf148a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:57.729920 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2e92c-1a36-4844-88f7-21de7bf148a0 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:57.730581 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2e92c-1a36-4844-88f7-21de7bf148a0 None admin] RBAC: Adding query filter params (name=image) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:57.730885 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2e92c-1a36-4844-88f7-21de7bf148a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:57.732350 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2e92c-1a36-4844-88f7-21de7bf148a0 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:57.737672 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 524/1061] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:57.743072 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2e92c-1a36-4844-88f7-21de7bf148a0 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:57.781576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b3555d-d86c-4ac8-9d4c-e862075681c3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:57.818173 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ccbf1a-3830-4497-a9d5-e674bdeeccb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:57.818746 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ccbf1a-3830-4497-a9d5-e674bdeeccb4 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:57.819027 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ccbf1a-3830-4497-a9d5-e674bdeeccb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:57.819302 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ccbf1a-3830-4497-a9d5-e674bdeeccb4 None admin] PATH_INFO: `/v3/projects/fda69e8c2d5e466dbc1244b2f76296a3/users/5e0fe2e84f6748c992c9beedf2f5d49b/roles/ae888494799042399af96f094898c4ed` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:57.821480 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ccbf1a-3830-4497-a9d5-e674bdeeccb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=fda69e8c2d5e466dbc1244b2f76296a3, user_id=5e0fe2e84f6748c992c9beedf2f5d49b, role_id=ae888494799042399af96f094898c4ed)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:57.822916 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ccbf1a-3830-4497-a9d5-e674bdeeccb4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:57.838806 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 538/1062] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 00:52:57 2022] PUT /identity/v3/projects/fda69e8c2d5e466dbc1244b2f76296a3/users/5e0fe2e84f6748c992c9beedf2f5d49b/roles/ae888494799042399af96f094898c4ed => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 00:52:57.850528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2e92c-1a36-4844-88f7-21de7bf148a0 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:57.878410 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:57.892711 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de195b59-ca16-40b5-a120-6b0827b53303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:57.893345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de195b59-ca16-40b5-a120-6b0827b53303 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:57.893631 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de195b59-ca16-40b5-a120-6b0827b53303 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:57.893907 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de195b59-ca16-40b5-a120-6b0827b53303 None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:57.894556 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de195b59-ca16-40b5-a120-6b0827b53303 None admin] RBAC: Adding query filter params (type=image) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:57.894874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de195b59-ca16-40b5-a120-6b0827b53303 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:57.896333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de195b59-ca16-40b5-a120-6b0827b53303 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:57.907718 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 525/1063] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:52:57 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:52:57.909155 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ccbf1a-3830-4497-a9d5-e674bdeeccb4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:58.022971 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ccbf1a-3830-4497-a9d5-e674bdeeccb4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:58.037027 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:58.063315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ccfc4f-1369-43ad-9c96-2cd9c5f9fc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:58.063972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccfc4f-1369-43ad-9c96-2cd9c5f9fc5b None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:58.064247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccfc4f-1369-43ad-9c96-2cd9c5f9fc5b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:58.064522 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccfc4f-1369-43ad-9c96-2cd9c5f9fc5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:58.065194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ccfc4f-1369-43ad-9c96-2cd9c5f9fc5b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:58.066560 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ccfc4f-1369-43ad-9c96-2cd9c5f9fc5b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:58.086983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 539/1064] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 00:52:57 2022] POST /identity/v3/endpoints => generated 327 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:52:58.678375 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f2f50-e684-4420-99d3-3f2d69fa9af5 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:58.678920 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f2f50-e684-4420-99d3-3f2d69fa9af5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:58.678920 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f2f50-e684-4420-99d3-3f2d69fa9af5 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:58.680332 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 526/1065] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:52:58.685927 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b224b36-becf-4133-bc9e-776fa6770b9a None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:58.686361 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b224b36-becf-4133-bc9e-776fa6770b9a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:58.686463 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b224b36-becf-4133-bc9e-776fa6770b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:58.704410 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0b224b36-becf-4133-bc9e-776fa6770b9a None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:52:58.705576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0b224b36-becf-4133-bc9e-776fa6770b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:52:58.780644 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 540/1066] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:52:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:52:59.050635 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f2f50-e684-4420-99d3-3f2d69fa9af5 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:59.155571 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f2f50-e684-4420-99d3-3f2d69fa9af5 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:59.189306 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c17c5-a00b-43a7-ae5d-cb383be6a664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:59.189934 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c17c5-a00b-43a7-ae5d-cb383be6a664 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:59.190217 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c17c5-a00b-43a7-ae5d-cb383be6a664 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:59.190486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c17c5-a00b-43a7-ae5d-cb383be6a664 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:59.202493 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c17c5-a00b-43a7-ae5d-cb383be6a664 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:52:59.207173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-da0c17c5-a00b-43a7-ae5d-cb383be6a664 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:52:59.207173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:52:59.207173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:52:59.207173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:52:59.207173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:52:59.207173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:52:59.207173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:52:59.207173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:52:59.207173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:52:59.207173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:52:59.207173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:52:59.207173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:52:59.207173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:52:59.207173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:52:59.207173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:52:59.207173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:52:59.208160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:52:59.208160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:52:59.208160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:52:59.208160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:52:59.208160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:52:59.208160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:52:59.208160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:52:59.208160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:52:59.208160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:52:59.208160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:52:59.208160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:52:59.208160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:52:59.208160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:52:59.208160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:52:59.208160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:52:59.208160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:52:59.209019 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:52:59.209019 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:52:59.209019 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:52:59.209019 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:52:59.209019 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:52:59.209019 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:52:59.209019 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:52:59.209019 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:52:59.209019 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:52:59.209019 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:52:59.209019 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:52:59.209019 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:52:59.211905 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 527/1067] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:52:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:52:59.214145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b224b36-becf-4133-bc9e-776fa6770b9a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:59.305453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:59.317359 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b224b36-becf-4133-bc9e-776fa6770b9a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:59.351728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa595a-781a-42f3-8cd1-2bc5b2456b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:59.352343 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa595a-781a-42f3-8cd1-2bc5b2456b51 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:59.352617 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa595a-781a-42f3-8cd1-2bc5b2456b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:59.352886 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa595a-781a-42f3-8cd1-2bc5b2456b51 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:59.353538 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fa595a-781a-42f3-8cd1-2bc5b2456b51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:52:59.353835 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fa595a-781a-42f3-8cd1-2bc5b2456b51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:59.355187 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fa595a-781a-42f3-8cd1-2bc5b2456b51 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:59.364288 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 541/1068] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:52:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:59.370237 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c17c5-a00b-43a7-ae5d-cb383be6a664 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:59.418631 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:59.478709 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c17c5-a00b-43a7-ae5d-cb383be6a664 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:59.511778 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581f68b-e7b8-4aa6-8bc0-c352fc5d3308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:59.512414 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581f68b-e7b8-4aa6-8bc0-c352fc5d3308 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:59.512692 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581f68b-e7b8-4aa6-8bc0-c352fc5d3308 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:59.512965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581f68b-e7b8-4aa6-8bc0-c352fc5d3308 None admin] PATH_INFO: `/v3/users/a24fb85125174b46acf2dd5e335e28da` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:59.527962 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581f68b-e7b8-4aa6-8bc0-c352fc5d3308 None admin] RBAC: Authorizing `identity:get_user(user_id=a24fb85125174b46acf2dd5e335e28da)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:59.530189 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581f68b-e7b8-4aa6-8bc0-c352fc5d3308 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:59.532209 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 528/1069] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:52:59 2022] GET /identity/v3/users/a24fb85125174b46acf2dd5e335e28da => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:52:59.537675 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa595a-781a-42f3-8cd1-2bc5b2456b51 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:59.640151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa595a-781a-42f3-8cd1-2bc5b2456b51 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:59.674043 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45a159-3366-4bda-9972-cfc1ed669d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:59.674648 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45a159-3366-4bda-9972-cfc1ed669d6d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:59.674924 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45a159-3366-4bda-9972-cfc1ed669d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:59.675191 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45a159-3366-4bda-9972-cfc1ed669d6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:59.699203 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:59.744863 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45a159-3366-4bda-9972-cfc1ed669d6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:59.746307 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45a159-3366-4bda-9972-cfc1ed669d6d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:52:59.845612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 542/1070] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:52:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 00:52:59.851280 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581f68b-e7b8-4aa6-8bc0-c352fc5d3308 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:52:59.959189 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581f68b-e7b8-4aa6-8bc0-c352fc5d3308 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:52:59.961935 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:52:59.995607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc8586-30d8-4627-aed3-b188b762bf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:52:59.996247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc8586-30d8-4627-aed3-b188b762bf15 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:52:59.996528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc8586-30d8-4627-aed3-b188b762bf15 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:52:59.996802 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc8586-30d8-4627-aed3-b188b762bf15 None admin] PATH_INFO: `/v3/projects/b8e96d6e59f84824b35d74726a4d7135` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:52:59.998486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc8586-30d8-4627-aed3-b188b762bf15 None admin] RBAC: Authorizing `identity:get_project(project_id=b8e96d6e59f84824b35d74726a4d7135)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:52:59.999969 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc8586-30d8-4627-aed3-b188b762bf15 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:00.006545 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 529/1071] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:52:59 2022] GET /identity/v3/projects/b8e96d6e59f84824b35d74726a4d7135 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:53:00.007794 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45a159-3366-4bda-9972-cfc1ed669d6d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:00.033364 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9429f9-6811-4cea-802e-3d50008001dd None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:00.033649 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9429f9-6811-4cea-802e-3d50008001dd None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:00.033931 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9429f9-6811-4cea-802e-3d50008001dd None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:00.035529 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 530/1072] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:53:00.040188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c629162-28d6-478f-a38e-ebd8e35b9fc9 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:00.040475 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c629162-28d6-478f-a38e-ebd8e35b9fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:00.040854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c629162-28d6-478f-a38e-ebd8e35b9fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:00.061203 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-4c629162-28d6-478f-a38e-ebd8e35b9fc9 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:53:00.062401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4c629162-28d6-478f-a38e-ebd8e35b9fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:53:00.070512 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:00.121866 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45a159-3366-4bda-9972-cfc1ed669d6d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:00.143763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 531/1073] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:53:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 00:53:00.160129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaade59-56b0-4fcf-ae60-df0835a7409b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:00.160750 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaade59-56b0-4fcf-ae60-df0835a7409b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:00.161029 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaade59-56b0-4fcf-ae60-df0835a7409b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:00.161301 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaade59-56b0-4fcf-ae60-df0835a7409b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:00.162019 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaade59-56b0-4fcf-ae60-df0835a7409b None admin] RBAC: Adding query filter params (role.id=ada268ea63354a0297070369dfe3c6cc, scope.project.id=b8e96d6e59f84824b35d74726a4d7135, user.id=a24fb85125174b46acf2dd5e335e28da) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:53:00.162325 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaade59-56b0-4fcf-ae60-df0835a7409b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:00.163766 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaade59-56b0-4fcf-ae60-df0835a7409b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:00.169388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 543/1074] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:53:00 2022] GET /identity/v3/role_assignments?user.id=a24fb85125174b46acf2dd5e335e28da&scope.project.id=b8e96d6e59f84824b35d74726a4d7135&role.id=ada268ea63354a0297070369dfe3c6cc => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:53:00.341119 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae650b0c-c579-445b-a5d1-e615646aaba4 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:00.341414 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae650b0c-c579-445b-a5d1-e615646aaba4 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:00.342257 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae650b0c-c579-445b-a5d1-e615646aaba4 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:00.343934 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 532/1075] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:53:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:53:00.349613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58143103-c9c1-4df2-a143-ce22cf9cc8c1 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:00.349903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58143103-c9c1-4df2-a143-ce22cf9cc8c1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:00.350187 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58143103-c9c1-4df2-a143-ce22cf9cc8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:00.369029 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-58143103-c9c1-4df2-a143-ce22cf9cc8c1 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:53:00.370216 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-58143103-c9c1-4df2-a143-ce22cf9cc8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:53:00.418549 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae650b0c-c579-445b-a5d1-e615646aaba4 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:00.447625 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 544/1076] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:53:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:53:00.530748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae650b0c-c579-445b-a5d1-e615646aaba4 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:00.532477 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:00.568004 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbaa32-d8fc-48b6-ad1a-ee2b17f975a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:00.568631 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbaa32-d8fc-48b6-ad1a-ee2b17f975a8 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:00.568970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbaa32-d8fc-48b6-ad1a-ee2b17f975a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:00.569248 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbaa32-d8fc-48b6-ad1a-ee2b17f975a8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:00.579798 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bbaa32-d8fc-48b6-ad1a-ee2b17f975a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:53:00.584465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-e1bbaa32-d8fc-48b6-ad1a-ee2b17f975a8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 00:53:00.584465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:53:00.584465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:53:00.584465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:53:00.584465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:53:00.584465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:53:00.584465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:53:00.584465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:53:00.584465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:53:00.584465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:53:00.584465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:53:00.584465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:53:00.584465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:53:00.584465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:53:00.584465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:53:00.584465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:53:00.585406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:53:00.585406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:53:00.585406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:53:00.585406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:53:00.585406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:53:00.585406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:53:00.585406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:53:00.585406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:53:00.585406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:53:00.585406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:53:00.585406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:53:00.585406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:53:00.585406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:53:00.585406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:53:00.585406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:53:00.585406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:53:00.588608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:53:00.588608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:53:00.588608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:53:00.588608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:53:00.588608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:53:00.588608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:53:00.588608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:53:00.588608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:53:00.588608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:53:00.588608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:53:00.588608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 00:53:00.588608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:53:00.588608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 533/1077] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Sep 14 00:53:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:53:00.591751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58143103-c9c1-4df2-a143-ce22cf9cc8c1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:00.643925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:00.696117 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58143103-c9c1-4df2-a143-ce22cf9cc8c1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:00.718436 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbaa32-d8fc-48b6-ad1a-ee2b17f975a8 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:00.733518 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb57159-7dc9-41b7-ad46-289acba5a276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:00.734138 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb57159-7dc9-41b7-ad46-289acba5a276 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:00.734419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb57159-7dc9-41b7-ad46-289acba5a276 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:00.734693 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb57159-7dc9-41b7-ad46-289acba5a276 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:00.735364 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb57159-7dc9-41b7-ad46-289acba5a276 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:53:00.735701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb57159-7dc9-41b7-ad46-289acba5a276 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:00.737073 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb57159-7dc9-41b7-ad46-289acba5a276 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:00.746347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 545/1078] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Sep 14 00:53:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:53:00.752521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb57159-7dc9-41b7-ad46-289acba5a276 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:00.827922 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbaa32-d8fc-48b6-ad1a-ee2b17f975a8 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:00.861261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb57159-7dc9-41b7-ad46-289acba5a276 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:00.863655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060dd07-816a-44c4-b3d7-49a22131895f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:00.864274 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060dd07-816a-44c4-b3d7-49a22131895f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:00.864554 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060dd07-816a-44c4-b3d7-49a22131895f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:00.864824 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060dd07-816a-44c4-b3d7-49a22131895f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:00.872487 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:00.897253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:00.901567 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3cccb-3317-4f87-8a60-fe8151b70569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:00.902367 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3cccb-3317-4f87-8a60-fe8151b70569 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:00.903412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3cccb-3317-4f87-8a60-fe8151b70569 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:00.903736 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3cccb-3317-4f87-8a60-fe8151b70569 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:00.917059 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed3cccb-3317-4f87-8a60-fe8151b70569 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:00.918977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed3cccb-3317-4f87-8a60-fe8151b70569 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:00.937132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-8ed3cccb-3317-4f87-8a60-fe8151b70569 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:53:00.937132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:53:00.937132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:53:00.937132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:53:00.937132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:53:00.937132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:53:00.937132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:53:00.937132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:53:00.937132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:53:00.937132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:53:00.937132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:53:00.937132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:53:00.937132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:53:00.937132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:53:00.937132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:53:00.937132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:53:00.938145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:53:00.938145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:53:00.938145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:53:00.938145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:53:00.938145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:53:00.938145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:53:00.938145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:53:00.938145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:53:00.938145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:53:00.938145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:53:00.938145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:53:00.938145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:53:00.938145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:53:00.938145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:53:00.938145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:53:00.938145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:53:00.938145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:53:00.939677 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:53:00.939677 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:53:00.939677 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:53:00.939857 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060dd07-816a-44c4-b3d7-49a22131895f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:00.939933 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 546/1079] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:53:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:53:00.940020 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060dd07-816a-44c4-b3d7-49a22131895f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:00.943831 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3cccb-3317-4f87-8a60-fe8151b70569 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:01.048748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 534/1080] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:53:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 14 00:53:01.053215 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3cccb-3317-4f87-8a60-fe8151b70569 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:01.054131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060dd07-816a-44c4-b3d7-49a22131895f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:01.090666 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b883ff84-1da3-4cf4-aca8-d56d0533d4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:01.092251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883ff84-1da3-4cf4-aca8-d56d0533d4f7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:01.092545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883ff84-1da3-4cf4-aca8-d56d0533d4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:01.093021 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883ff84-1da3-4cf4-aca8-d56d0533d4f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:01.093790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883ff84-1da3-4cf4-aca8-d56d0533d4f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:53:01.094157 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883ff84-1da3-4cf4-aca8-d56d0533d4f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:01.095837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883ff84-1da3-4cf4-aca8-d56d0533d4f7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:01.108586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 547/1081] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:53:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:53:01.114028 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b883ff84-1da3-4cf4-aca8-d56d0533d4f7 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:01.125236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:01.139440 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:01.168402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060dd07-816a-44c4-b3d7-49a22131895f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:01.207697 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e32bff-b4db-4321-b51d-85f6a24ae3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:01.208289 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e32bff-b4db-4321-b51d-85f6a24ae3ab None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:01.208575 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e32bff-b4db-4321-b51d-85f6a24ae3ab None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:01.208846 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e32bff-b4db-4321-b51d-85f6a24ae3ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:01.222404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e32bff-b4db-4321-b51d-85f6a24ae3ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:01.224119 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e32bff-b4db-4321-b51d-85f6a24ae3ab None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:01.224923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b883ff84-1da3-4cf4-aca8-d56d0533d4f7 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:01.240064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-66e32bff-b4db-4321-b51d-85f6a24ae3ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:53:01.240064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:53:01.240064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:53:01.240064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:53:01.240064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:53:01.240064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:53:01.240064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:53:01.240064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:53:01.240064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:53:01.240064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:53:01.240064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:53:01.240064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:53:01.240064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:53:01.240064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:53:01.240064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:53:01.240064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:53:01.241129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:53:01.241129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:53:01.241129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:53:01.241129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:53:01.241129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:53:01.241129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:53:01.241129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:53:01.241129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:53:01.241129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:53:01.241129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:53:01.241129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:53:01.241129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:53:01.241129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:53:01.241129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:53:01.241129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:53:01.241129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:53:01.241129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:53:01.242125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:53:01.242125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:53:01.242125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:53:01.243976 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 535/1082] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:53:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:53:01.247382 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e32bff-b4db-4321-b51d-85f6a24ae3ab None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:01.267522 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03f7e1-1dd0-4e48-901a-af2fb2f80feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:01.268185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03f7e1-1dd0-4e48-901a-af2fb2f80feb None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:01.268468 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03f7e1-1dd0-4e48-901a-af2fb2f80feb None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:01.268745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03f7e1-1dd0-4e48-901a-af2fb2f80feb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:01.276177 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03f7e1-1dd0-4e48-901a-af2fb2f80feb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:01.277974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03f7e1-1dd0-4e48-901a-af2fb2f80feb None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:01.292928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-7b03f7e1-1dd0-4e48-901a-af2fb2f80feb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 00:53:01.292928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:53:01.292928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:53:01.292928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:53:01.292928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:53:01.292928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:53:01.292928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:53:01.292928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:53:01.292928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:53:01.292928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:53:01.292928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:53:01.292928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:53:01.292928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:53:01.292928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:53:01.292928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:53:01.292928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:53:01.293931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:53:01.293931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:53:01.293931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:53:01.293931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:53:01.293931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:53:01.293931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:53:01.293931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:53:01.293931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:53:01.293931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:53:01.293931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:53:01.293931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:53:01.293931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:53:01.293931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:53:01.293931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:53:01.293931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:53:01.293931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:53:01.293931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:53:01.293931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:53:01.300715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:53:01.300715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:53:01.300715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:53:01.300715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:53:01.300715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:53:01.300715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:53:01.300715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:53:01.300715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:53:01.300715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 00:53:01.300715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:53:01.300715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 548/1083] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Sep 14 00:53:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:53:01.300715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03f7e1-1dd0-4e48-901a-af2fb2f80feb None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:01.361872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e32bff-b4db-4321-b51d-85f6a24ae3ab None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:01.379273 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:01.393958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:01.401464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f710872-d5fe-4454-a486-bd1b367b34b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:01.402088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f710872-d5fe-4454-a486-bd1b367b34b0 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:01.402370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f710872-d5fe-4454-a486-bd1b367b34b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:01.402646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f710872-d5fe-4454-a486-bd1b367b34b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:01.403303 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f710872-d5fe-4454-a486-bd1b367b34b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:53:01.403637 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f710872-d5fe-4454-a486-bd1b367b34b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:01.405113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f710872-d5fe-4454-a486-bd1b367b34b0 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:01.415483 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03f7e1-1dd0-4e48-901a-af2fb2f80feb None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:01.418151 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 536/1084] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:53:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:53:01.455399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a4bc8-634f-45de-bfa9-efa3d4e51a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:01.456071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a4bc8-634f-45de-bfa9-efa3d4e51a16 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:01.456356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a4bc8-634f-45de-bfa9-efa3d4e51a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:01.456632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a4bc8-634f-45de-bfa9-efa3d4e51a16 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:01.457387 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a4bc8-634f-45de-bfa9-efa3d4e51a16 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:53:01.457701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a4bc8-634f-45de-bfa9-efa3d4e51a16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:01.459198 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a4bc8-634f-45de-bfa9-efa3d4e51a16 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:01.492311 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 549/1085] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Sep 14 00:53:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:01.493338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f710872-d5fe-4454-a486-bd1b367b34b0 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:01.604659 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f710872-d5fe-4454-a486-bd1b367b34b0 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:01.641087 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f5b72-2b8f-4546-a036-2f4bc241cca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:01.641634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f5b72-2b8f-4546-a036-2f4bc241cca2 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:01.641918 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f5b72-2b8f-4546-a036-2f4bc241cca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:01.642193 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f5b72-2b8f-4546-a036-2f4bc241cca2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:01.687425 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:01.717581 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f5b72-2b8f-4546-a036-2f4bc241cca2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:01.719056 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f5b72-2b8f-4546-a036-2f4bc241cca2 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:01.819990 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 537/1086] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:53:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 00:53:01.825790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a4bc8-634f-45de-bfa9-efa3d4e51a16 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:01.924034 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a4bc8-634f-45de-bfa9-efa3d4e51a16 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:01.957632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b13b44-f617-4796-8722-5dcf5121a922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:01.958263 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b13b44-f617-4796-8722-5dcf5121a922 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:01.958545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b13b44-f617-4796-8722-5dcf5121a922 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:01.958818 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b13b44-f617-4796-8722-5dcf5121a922 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:01.966920 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:01.969415 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b13b44-f617-4796-8722-5dcf5121a922 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:01.971181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b13b44-f617-4796-8722-5dcf5121a922 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:01.986132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-23b13b44-f617-4796-8722-5dcf5121a922 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:53:01.986132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:53:01.986132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:53:01.986132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:53:01.986132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:53:01.986132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:53:01.986132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:53:01.986132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:53:01.986132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:53:01.986132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:53:01.986132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:53:01.986132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:53:01.986132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:53:01.986132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:53:01.986132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:53:01.986132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:53:01.987206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:53:01.987206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:53:01.987206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:53:01.987206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:53:01.987206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:53:01.987206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:53:01.987206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:53:01.987206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:53:01.987206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:53:01.987206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:53:01.987206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:53:01.987206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:53:01.987206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:53:01.987206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:53:01.987206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:53:01.987206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:53:01.987206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:53:01.988187 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:53:01.988187 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:53:01.988187 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:53:01.988187 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 550/1087] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:53:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:53:01.993133 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f5b72-2b8f-4546-a036-2f4bc241cca2 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:02.067135 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:02.092477 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f5b72-2b8f-4546-a036-2f4bc241cca2 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:02.127244 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3760de-de62-4de5-b1f7-9c9eae37f969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:02.127885 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3760de-de62-4de5-b1f7-9c9eae37f969 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:02.128165 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3760de-de62-4de5-b1f7-9c9eae37f969 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:02.128437 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3760de-de62-4de5-b1f7-9c9eae37f969 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:02.129074 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3760de-de62-4de5-b1f7-9c9eae37f969 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:53:02.129377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3760de-de62-4de5-b1f7-9c9eae37f969 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:02.130739 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3760de-de62-4de5-b1f7-9c9eae37f969 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:02.146971 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 538/1088] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:53:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:02.148195 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b13b44-f617-4796-8722-5dcf5121a922 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:02.250849 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b13b44-f617-4796-8722-5dcf5121a922 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:02.285475 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e0c1a-7cbd-4821-a23c-7de1e74c4d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:02.286102 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e0c1a-7cbd-4821-a23c-7de1e74c4d57 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:02.286382 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e0c1a-7cbd-4821-a23c-7de1e74c4d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:02.286653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e0c1a-7cbd-4821-a23c-7de1e74c4d57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:02.293088 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e0c1a-7cbd-4821-a23c-7de1e74c4d57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:02.295139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e0c1a-7cbd-4821-a23c-7de1e74c4d57 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:02.305979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-5e1e0c1a-7cbd-4821-a23c-7de1e74c4d57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:53:02.305979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:53:02.305979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:53:02.305979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:53:02.305979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:53:02.305979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:53:02.305979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:53:02.305979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:53:02.305979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:53:02.305979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:53:02.305979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:53:02.305979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:53:02.305979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:53:02.305979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:53:02.305979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:53:02.306936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:53:02.306936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:53:02.306936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:53:02.306936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:53:02.306936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:53:02.306936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:53:02.306936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:53:02.306936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:53:02.306936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:53:02.306936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:53:02.306936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:53:02.306936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:53:02.306936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:53:02.306936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:53:02.306936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:53:02.306936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:53:02.306936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:53:02.308248 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:53:02.308248 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:53:02.308248 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:53:02.308248 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:53:02.308248 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:53:02.308248 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:53:02.308248 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:53:02.308248 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:53:02.308248 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 551/1089] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:53:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:53:02.313528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3760de-de62-4de5-b1f7-9c9eae37f969 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:02.351999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba46e14-e651-4083-a248-25e15d30d6da None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:02.352281 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba46e14-e651-4083-a248-25e15d30d6da None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:02.352557 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba46e14-e651-4083-a248-25e15d30d6da None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:02.353876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 552/1090] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:53:02.359492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff941b-cfc1-4675-afae-50f4cf10a3ad None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:02.359807 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff941b-cfc1-4675-afae-50f4cf10a3ad None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:02.360088 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff941b-cfc1-4675-afae-50f4cf10a3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:02.378629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-85ff941b-cfc1-4675-afae-50f4cf10a3ad None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:53:02.379843 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-85ff941b-cfc1-4675-afae-50f4cf10a3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:53:02.413652 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3760de-de62-4de5-b1f7-9c9eae37f969 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:02.427983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:02.446909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cb0ca-ead2-479e-8d23-d5379bd56acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:02.447512 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cb0ca-ead2-479e-8d23-d5379bd56acf None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:02.447637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 553/1091] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:53:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:53:02.447887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cb0ca-ead2-479e-8d23-d5379bd56acf None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:02.448164 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cb0ca-ead2-479e-8d23-d5379bd56acf None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:02.448915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297cb0ca-ead2-479e-8d23-d5379bd56acf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:53:02.449218 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297cb0ca-ead2-479e-8d23-d5379bd56acf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:02.450636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297cb0ca-ead2-479e-8d23-d5379bd56acf None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:02.452609 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:02.463665 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 539/1092] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:53:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:02.470698 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff941b-cfc1-4675-afae-50f4cf10a3ad None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:02.567440 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff941b-cfc1-4675-afae-50f4cf10a3ad None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:02.600441 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471738e7-80c8-4d02-a163-3bf80f1de066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:02.601047 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471738e7-80c8-4d02-a163-3bf80f1de066 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:02.601331 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471738e7-80c8-4d02-a163-3bf80f1de066 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:02.601604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471738e7-80c8-4d02-a163-3bf80f1de066 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:02.602323 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471738e7-80c8-4d02-a163-3bf80f1de066 None admin] RBAC: Adding query filter params (role.id=ae888494799042399af96f094898c4ed, scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3, user.id=5e0fe2e84f6748c992c9beedf2f5d49b) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:53:02.602634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471738e7-80c8-4d02-a163-3bf80f1de066 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:02.604093 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471738e7-80c8-4d02-a163-3bf80f1de066 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:02.610302 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 554/1093] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:53:02 2022] GET /identity/v3/role_assignments?user.id=5e0fe2e84f6748c992c9beedf2f5d49b&scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3&role.id=ae888494799042399af96f094898c4ed => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:53:02.722444 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cb0ca-ead2-479e-8d23-d5379bd56acf None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:02.826610 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cb0ca-ead2-479e-8d23-d5379bd56acf None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:02.861346 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89298848-adde-4344-a88b-7ea8d6cdd2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:02.861983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89298848-adde-4344-a88b-7ea8d6cdd2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:02.862260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89298848-adde-4344-a88b-7ea8d6cdd2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:02.862529 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89298848-adde-4344-a88b-7ea8d6cdd2f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:02.873538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89298848-adde-4344-a88b-7ea8d6cdd2f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:53:02.878389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-89298848-adde-4344-a88b-7ea8d6cdd2f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:53:02.878389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:53:02.878389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:53:02.878389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:53:02.878389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:53:02.878389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:53:02.878389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:53:02.878389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:53:02.878389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:53:02.878389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:53:02.878389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:53:02.878389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:53:02.878389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:53:02.878389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:53:02.878389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:53:02.878389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:53:02.879590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:53:02.879590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:53:02.879590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:53:02.879590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:53:02.879590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:53:02.879590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:53:02.879590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:53:02.879590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:53:02.879590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:53:02.879590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:53:02.879590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:53:02.879590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:53:02.879590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:53:02.879590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:53:02.879590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:53:02.879590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:53:02.880736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:53:02.880736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:53:02.880736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:53:02.880736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:53:02.880736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:53:02.880736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:53:02.880736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:53:02.880736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:53:02.880736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:53:02.880736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:53:02.880736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:53:02.880736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:53:02.880736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 540/1094] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:53:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:53:02.886700 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471738e7-80c8-4d02-a163-3bf80f1de066 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:02.939393 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:02.995718 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471738e7-80c8-4d02-a163-3bf80f1de066 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:03.031014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831deea8-b8e7-4aa1-ad9f-9d8bb246d61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:03.031662 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831deea8-b8e7-4aa1-ad9f-9d8bb246d61b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:03.031950 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831deea8-b8e7-4aa1-ad9f-9d8bb246d61b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:03.032227 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831deea8-b8e7-4aa1-ad9f-9d8bb246d61b None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:03.032890 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831deea8-b8e7-4aa1-ad9f-9d8bb246d61b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:53:03.033196 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831deea8-b8e7-4aa1-ad9f-9d8bb246d61b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:03.034568 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831deea8-b8e7-4aa1-ad9f-9d8bb246d61b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:03.042834 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 555/1095] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:53:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:53:03.049462 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89298848-adde-4344-a88b-7ea8d6cdd2f1 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:03.113815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:03.161277 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89298848-adde-4344-a88b-7ea8d6cdd2f1 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:03.196896 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895bb7f-5100-477f-8622-f2770d8d46fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:03.197525 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895bb7f-5100-477f-8622-f2770d8d46fb None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:03.197809 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895bb7f-5100-477f-8622-f2770d8d46fb None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:03.198086 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895bb7f-5100-477f-8622-f2770d8d46fb None admin] PATH_INFO: `/v3/users/a24fb85125174b46acf2dd5e335e28da` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:03.199204 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895bb7f-5100-477f-8622-f2770d8d46fb None admin] RBAC: Authorizing `identity:get_user(user_id=a24fb85125174b46acf2dd5e335e28da)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:03.200702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895bb7f-5100-477f-8622-f2770d8d46fb None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:03.202464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 541/1096] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:53:03 2022] GET /identity/v3/users/a24fb85125174b46acf2dd5e335e28da => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:03.208393 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831deea8-b8e7-4aa1-ad9f-9d8bb246d61b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:03.314359 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831deea8-b8e7-4aa1-ad9f-9d8bb246d61b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:03.323352 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:03.350153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c8afb-4b46-441b-8e65-d6d1c3508782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:03.350793 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c8afb-4b46-441b-8e65-d6d1c3508782 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:03.351074 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c8afb-4b46-441b-8e65-d6d1c3508782 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:03.351350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c8afb-4b46-441b-8e65-d6d1c3508782 None admin] PATH_INFO: `/v3/projects/b8e96d6e59f84824b35d74726a4d7135` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:03.352502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c8afb-4b46-441b-8e65-d6d1c3508782 None admin] RBAC: Authorizing `identity:get_project(project_id=b8e96d6e59f84824b35d74726a4d7135)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:03.353934 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c8afb-4b46-441b-8e65-d6d1c3508782 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:03.355624 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 556/1097] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:53:03 2022] GET /identity/v3/projects/b8e96d6e59f84824b35d74726a4d7135 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:03.362565 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895bb7f-5100-477f-8622-f2770d8d46fb None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:03.464839 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895bb7f-5100-477f-8622-f2770d8d46fb None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:03.493110 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:03.506153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076b7a5-62ca-4edd-ac14-3bd5a04d6872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:03.506824 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076b7a5-62ca-4edd-ac14-3bd5a04d6872 None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:03.507106 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076b7a5-62ca-4edd-ac14-3bd5a04d6872 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:03.507382 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076b7a5-62ca-4edd-ac14-3bd5a04d6872 None admin] PATH_INFO: `/v3/projects/b8e96d6e59f84824b35d74726a4d7135/users/a24fb85125174b46acf2dd5e335e28da/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:03.509384 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076b7a5-62ca-4edd-ac14-3bd5a04d6872 None admin] RBAC: Authorizing `identity:create_grant(project_id=b8e96d6e59f84824b35d74726a4d7135, user_id=a24fb85125174b46acf2dd5e335e28da, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:03.510858 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076b7a5-62ca-4edd-ac14-3bd5a04d6872 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:03.523498 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 542/1098] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 00:53:03 2022] PUT /identity/v3/projects/b8e96d6e59f84824b35d74726a4d7135/users/a24fb85125174b46acf2dd5e335e28da/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 00:53:05.643034 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0cfe4-a66f-4d14-9e41-658001d0d1ae None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:05.643613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0cfe4-a66f-4d14-9e41-658001d0d1ae None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:05.643613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0cfe4-a66f-4d14-9e41-658001d0d1ae None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:05.644947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 557/1099] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:53:05.650071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2eff0-ad74-4ef4-8007-091ee574f6a2 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:05.650461 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2eff0-ad74-4ef4-8007-091ee574f6a2 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:05.650628 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2eff0-ad74-4ef4-8007-091ee574f6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:05.668877 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-c7b2eff0-ad74-4ef4-8007-091ee574f6a2 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:53:05.670327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c7b2eff0-ad74-4ef4-8007-091ee574f6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:53:05.734576 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 543/1100] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:53:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:53:06.007063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0cfe4-a66f-4d14-9e41-658001d0d1ae None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:06.110545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0cfe4-a66f-4d14-9e41-658001d0d1ae None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:06.144638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68201a71-e01e-43b5-9ae2-ce02846edab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:06.145252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68201a71-e01e-43b5-9ae2-ce02846edab0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:06.145527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68201a71-e01e-43b5-9ae2-ce02846edab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:06.145796 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68201a71-e01e-43b5-9ae2-ce02846edab0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:06.156341 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68201a71-e01e-43b5-9ae2-ce02846edab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:53:06.161065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-68201a71-e01e-43b5-9ae2-ce02846edab0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:53:06.161065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:53:06.161065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:53:06.161065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:53:06.161065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:53:06.161065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:53:06.161065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:53:06.161065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:53:06.161065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:53:06.161065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:53:06.161065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:53:06.161065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:53:06.161065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:53:06.161065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:53:06.161065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:53:06.161065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:53:06.162230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:53:06.162230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:53:06.162230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:53:06.162230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:53:06.162230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:53:06.162230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:53:06.162230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:53:06.162230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:53:06.162230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:53:06.162230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:53:06.162230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:53:06.162230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:53:06.162230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:53:06.162230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:53:06.162230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:53:06.162230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:53:06.163520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:53:06.163520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:53:06.163520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:53:06.163520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:53:06.163520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:53:06.163520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:53:06.163520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:53:06.163520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:53:06.163520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:53:06.163520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:53:06.163520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:53:06.163520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:53:06.163520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 558/1101] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:53:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:53:06.168155 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2eff0-ad74-4ef4-8007-091ee574f6a2 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:06.266409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2eff0-ad74-4ef4-8007-091ee574f6a2 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:06.279409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:06.309327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf3f63-b20b-4970-bf6b-87267c051258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:06.309949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf3f63-b20b-4970-bf6b-87267c051258 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:06.310402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf3f63-b20b-4970-bf6b-87267c051258 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:06.310821 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf3f63-b20b-4970-bf6b-87267c051258 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:06.311480 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cf3f63-b20b-4970-bf6b-87267c051258 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:53:06.311811 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cf3f63-b20b-4970-bf6b-87267c051258 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:06.313184 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cf3f63-b20b-4970-bf6b-87267c051258 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:06.321255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 544/1102] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:53:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:06.327375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68201a71-e01e-43b5-9ae2-ce02846edab0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:06.362894 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:06.427490 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68201a71-e01e-43b5-9ae2-ce02846edab0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:06.462596 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e9c9a-25a3-4561-a95f-9f208195226e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:06.463253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e9c9a-25a3-4561-a95f-9f208195226e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:06.463533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e9c9a-25a3-4561-a95f-9f208195226e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:06.463991 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e9c9a-25a3-4561-a95f-9f208195226e None admin] PATH_INFO: `/v3/users/a24fb85125174b46acf2dd5e335e28da` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:06.465119 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e9c9a-25a3-4561-a95f-9f208195226e None admin] RBAC: Authorizing `identity:get_user(user_id=a24fb85125174b46acf2dd5e335e28da)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:06.467085 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e9c9a-25a3-4561-a95f-9f208195226e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:06.468696 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 559/1103] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:53:06 2022] GET /identity/v3/users/a24fb85125174b46acf2dd5e335e28da => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:06.474457 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf3f63-b20b-4970-bf6b-87267c051258 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:06.573311 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf3f63-b20b-4970-bf6b-87267c051258 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:06.606246 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd87989-9e32-4a1d-b58b-fad42f18f56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:06.606908 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd87989-9e32-4a1d-b58b-fad42f18f56d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:06.607192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd87989-9e32-4a1d-b58b-fad42f18f56d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:06.607504 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd87989-9e32-4a1d-b58b-fad42f18f56d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:06.648270 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:06.673584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd87989-9e32-4a1d-b58b-fad42f18f56d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:06.675033 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd87989-9e32-4a1d-b58b-fad42f18f56d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:06.770327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 545/1104] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:53:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 00:53:06.776209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e9c9a-25a3-4561-a95f-9f208195226e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:06.871839 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e9c9a-25a3-4561-a95f-9f208195226e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:06.874609 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:06.906027 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f47d2-7eb6-4306-9445-8b832c646ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:06.906649 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f47d2-7eb6-4306-9445-8b832c646ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:06.906931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f47d2-7eb6-4306-9445-8b832c646ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:06.907207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f47d2-7eb6-4306-9445-8b832c646ad5 None admin] PATH_INFO: `/v3/projects/b8e96d6e59f84824b35d74726a4d7135` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:06.908350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f47d2-7eb6-4306-9445-8b832c646ad5 None admin] RBAC: Authorizing `identity:get_project(project_id=b8e96d6e59f84824b35d74726a4d7135)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:06.909783 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f47d2-7eb6-4306-9445-8b832c646ad5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:06.911383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 560/1105] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:53:06 2022] GET /identity/v3/projects/b8e96d6e59f84824b35d74726a4d7135 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:06.917435 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd87989-9e32-4a1d-b58b-fad42f18f56d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:06.990592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:07.014748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd87989-9e32-4a1d-b58b-fad42f18f56d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:07.048422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7837a54c-33f9-4cf7-89bf-8da51d87e21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:07.049013 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837a54c-33f9-4cf7-89bf-8da51d87e21f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:07.049296 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837a54c-33f9-4cf7-89bf-8da51d87e21f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:07.049574 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837a54c-33f9-4cf7-89bf-8da51d87e21f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:07.050285 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7837a54c-33f9-4cf7-89bf-8da51d87e21f None admin] RBAC: Adding query filter params (role.id=ada268ea63354a0297070369dfe3c6cc, scope.project.id=b8e96d6e59f84824b35d74726a4d7135, user.id=a24fb85125174b46acf2dd5e335e28da) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:53:07.050595 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7837a54c-33f9-4cf7-89bf-8da51d87e21f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:07.052056 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7837a54c-33f9-4cf7-89bf-8da51d87e21f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:07.057538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 546/1106] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:53:06 2022] GET /identity/v3/role_assignments?user.id=a24fb85125174b46acf2dd5e335e28da&scope.project.id=b8e96d6e59f84824b35d74726a4d7135&role.id=ada268ea63354a0297070369dfe3c6cc => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:09.211420 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a046e-c751-49c6-a037-206718e8047c None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:09.212009 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a046e-c751-49c6-a037-206718e8047c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:09.212009 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a046e-c751-49c6-a037-206718e8047c None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:09.213368 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 561/1107] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:53:09.219013 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495382e-4cd4-4143-87dc-ac57359a996c None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:09.219459 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495382e-4cd4-4143-87dc-ac57359a996c None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:09.219581 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495382e-4cd4-4143-87dc-ac57359a996c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:09.237996 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d495382e-4cd4-4143-87dc-ac57359a996c None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:53:09.239180 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d495382e-4cd4-4143-87dc-ac57359a996c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:53:09.303339 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 547/1108] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:53:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:53:09.574050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a046e-c751-49c6-a037-206718e8047c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:09.673938 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a046e-c751-49c6-a037-206718e8047c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:09.707531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6148e7-46fc-4bf0-97b0-ff8c940d33de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:09.708174 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6148e7-46fc-4bf0-97b0-ff8c940d33de None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:09.708485 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6148e7-46fc-4bf0-97b0-ff8c940d33de None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:09.708760 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6148e7-46fc-4bf0-97b0-ff8c940d33de None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:09.709860 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6148e7-46fc-4bf0-97b0-ff8c940d33de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:09.711287 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6148e7-46fc-4bf0-97b0-ff8c940d33de None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:09.712840 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 562/1109] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:53:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:09.718855 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495382e-4cd4-4143-87dc-ac57359a996c None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:09.784091 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:09.816887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495382e-4cd4-4143-87dc-ac57359a996c None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:09.849365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1911f-a5d3-404b-a01e-fd2851900f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:09.849963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1911f-a5d3-404b-a01e-fd2851900f8d None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:09.850245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1911f-a5d3-404b-a01e-fd2851900f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:09.850519 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1911f-a5d3-404b-a01e-fd2851900f8d None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:09.851173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1911f-a5d3-404b-a01e-fd2851900f8d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:09.852637 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1911f-a5d3-404b-a01e-fd2851900f8d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:09.874414 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 548/1110] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 00:53:09 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:53:12.054298 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d9a8d-a259-4af6-836f-3e2aad093126 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:12.054857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d9a8d-a259-4af6-836f-3e2aad093126 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:12.054857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d9a8d-a259-4af6-836f-3e2aad093126 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:12.056190 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 563/1111] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:53:12.062065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721c11d-a4a8-4895-a457-4dd0b2ba6a71 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:12.062468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721c11d-a4a8-4895-a457-4dd0b2ba6a71 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:12.062619 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721c11d-a4a8-4895-a457-4dd0b2ba6a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:12.080440 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-2721c11d-a4a8-4895-a457-4dd0b2ba6a71 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:53:12.081614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2721c11d-a4a8-4895-a457-4dd0b2ba6a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:53:12.146203 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 549/1112] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:53:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:53:12.413333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d9a8d-a259-4af6-836f-3e2aad093126 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:12.455537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:12.516047 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d9a8d-a259-4af6-836f-3e2aad093126 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:12.549461 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e550d6-2a5d-4045-adff-a6c80ada41ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:12.550082 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e550d6-2a5d-4045-adff-a6c80ada41ca None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:12.550363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e550d6-2a5d-4045-adff-a6c80ada41ca None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:12.550636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e550d6-2a5d-4045-adff-a6c80ada41ca None admin] PATH_INFO: `/v3/roles/89a3bc954b0a45bfa40a16db0ad2e51c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:12.552021 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e550d6-2a5d-4045-adff-a6c80ada41ca None admin] RBAC: Authorizing `identity:get_role(role_id=89a3bc954b0a45bfa40a16db0ad2e51c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:12.553375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e550d6-2a5d-4045-adff-a6c80ada41ca None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:12.554669 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 564/1113] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:53:12 2022] GET /identity/v3/roles/89a3bc954b0a45bfa40a16db0ad2e51c => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:12.560587 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721c11d-a4a8-4895-a457-4dd0b2ba6a71 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:12.613586 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:12.657916 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721c11d-a4a8-4895-a457-4dd0b2ba6a71 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:12.689876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51146f2a-1135-4556-a944-b09e23cf6686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:12.690440 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51146f2a-1135-4556-a944-b09e23cf6686 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:12.690721 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51146f2a-1135-4556-a944-b09e23cf6686 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:12.690996 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51146f2a-1135-4556-a944-b09e23cf6686 None admin] PATH_INFO: `/v3/users/32b5d85296d747b19042816650478bb2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:12.704035 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51146f2a-1135-4556-a944-b09e23cf6686 None admin] RBAC: Authorizing `identity:get_user(user_id=32b5d85296d747b19042816650478bb2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:12.705469 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51146f2a-1135-4556-a944-b09e23cf6686 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:12.707044 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 550/1114] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:53:12 2022] GET /identity/v3/users/32b5d85296d747b19042816650478bb2 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:12.712861 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e550d6-2a5d-4045-adff-a6c80ada41ca None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:12.808916 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e550d6-2a5d-4045-adff-a6c80ada41ca None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:12.817721 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:12.842846 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb6fcc-f519-4156-a2f7-5b0946c3da5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:12.843437 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb6fcc-f519-4156-a2f7-5b0946c3da5e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:12.843745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb6fcc-f519-4156-a2f7-5b0946c3da5e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:12.844019 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb6fcc-f519-4156-a2f7-5b0946c3da5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:12.908765 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb6fcc-f519-4156-a2f7-5b0946c3da5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:12.910198 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb6fcc-f519-4156-a2f7-5b0946c3da5e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:13.003892 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 565/1115] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:53:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 00:53:13.009772 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51146f2a-1135-4556-a944-b09e23cf6686 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:13.106400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51146f2a-1135-4556-a944-b09e23cf6686 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:13.120292 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:13.141625 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99fb23-21df-4a84-a9c8-60523d60142f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:13.142248 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99fb23-21df-4a84-a9c8-60523d60142f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:13.142529 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99fb23-21df-4a84-a9c8-60523d60142f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:13.142809 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99fb23-21df-4a84-a9c8-60523d60142f None admin] PATH_INFO: `/v3/projects/b8e96d6e59f84824b35d74726a4d7135` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:13.143939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac99fb23-21df-4a84-a9c8-60523d60142f None admin] RBAC: Authorizing `identity:get_project(project_id=b8e96d6e59f84824b35d74726a4d7135)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:13.145365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac99fb23-21df-4a84-a9c8-60523d60142f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:13.146971 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 551/1116] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:53:13 2022] GET /identity/v3/projects/b8e96d6e59f84824b35d74726a4d7135 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:13.153211 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb6fcc-f519-4156-a2f7-5b0946c3da5e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:13.163642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:13.250726 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb6fcc-f519-4156-a2f7-5b0946c3da5e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:13.283978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfbaa6-2afd-4c46-8ed6-83877e166de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:13.284508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfbaa6-2afd-4c46-8ed6-83877e166de8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:13.284782 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfbaa6-2afd-4c46-8ed6-83877e166de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:13.285049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfbaa6-2afd-4c46-8ed6-83877e166de8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:13.285754 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdfbaa6-2afd-4c46-8ed6-83877e166de8 None admin] RBAC: Adding query filter params (role.id=89a3bc954b0a45bfa40a16db0ad2e51c, scope.project.id=b8e96d6e59f84824b35d74726a4d7135, user.id=32b5d85296d747b19042816650478bb2) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:53:13.286056 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdfbaa6-2afd-4c46-8ed6-83877e166de8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:13.287460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdfbaa6-2afd-4c46-8ed6-83877e166de8 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:13.292697 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 566/1117] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:53:13 2022] GET /identity/v3/role_assignments?user.id=32b5d85296d747b19042816650478bb2&scope.project.id=b8e96d6e59f84824b35d74726a4d7135&role.id=89a3bc954b0a45bfa40a16db0ad2e51c => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:15.432828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a948f51-c116-4bb2-b519-6dc0522b3e40 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:15.433391 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a948f51-c116-4bb2-b519-6dc0522b3e40 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:15.433391 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a948f51-c116-4bb2-b519-6dc0522b3e40 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:15.435254 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 552/1118] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:53:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:53:15.440529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6736af-e1d8-47ce-8c3b-d001d839a1c6 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:15.440935 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6736af-e1d8-47ce-8c3b-d001d839a1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:15.441087 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6736af-e1d8-47ce-8c3b-d001d839a1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:15.459788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-3c6736af-e1d8-47ce-8c3b-d001d839a1c6 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:53:15.460978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3c6736af-e1d8-47ce-8c3b-d001d839a1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:53:15.526798 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 567/1119] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:53:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:53:15.800737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a948f51-c116-4bb2-b519-6dc0522b3e40 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:15.900506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a948f51-c116-4bb2-b519-6dc0522b3e40 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:15.933496 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a08071-3b59-4c41-b60c-536b54f04297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:15.934108 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a08071-3b59-4c41-b60c-536b54f04297 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:15.934385 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a08071-3b59-4c41-b60c-536b54f04297 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:15.934681 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a08071-3b59-4c41-b60c-536b54f04297 None admin] PATH_INFO: `/v3/roles/89a3bc954b0a45bfa40a16db0ad2e51c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:15.936074 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a08071-3b59-4c41-b60c-536b54f04297 None admin] RBAC: Authorizing `identity:get_role(role_id=89a3bc954b0a45bfa40a16db0ad2e51c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:15.937457 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a08071-3b59-4c41-b60c-536b54f04297 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:15.938747 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 553/1120] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:53:15 2022] GET /identity/v3/roles/89a3bc954b0a45bfa40a16db0ad2e51c => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:53:15.944524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6736af-e1d8-47ce-8c3b-d001d839a1c6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:15.946688 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:16.042290 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6736af-e1d8-47ce-8c3b-d001d839a1c6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:16.075366 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7c680-fd3e-4e12-bce8-9113e882d7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:16.075955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7c680-fd3e-4e12-bce8-9113e882d7b7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:16.076230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7c680-fd3e-4e12-bce8-9113e882d7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:16.076499 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7c680-fd3e-4e12-bce8-9113e882d7b7 None admin] PATH_INFO: `/v3/users/32b5d85296d747b19042816650478bb2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:16.077595 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef7c680-fd3e-4e12-bce8-9113e882d7b7 None admin] RBAC: Authorizing `identity:get_user(user_id=32b5d85296d747b19042816650478bb2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:16.079016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef7c680-fd3e-4e12-bce8-9113e882d7b7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:16.080569 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 568/1121] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:53:15 2022] GET /identity/v3/users/32b5d85296d747b19042816650478bb2 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:16.086237 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a08071-3b59-4c41-b60c-536b54f04297 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:16.135173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:16.185207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a08071-3b59-4c41-b60c-536b54f04297 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:16.218282 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4310e7-8e9c-4b7c-bb29-0e5162784fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:16.218895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4310e7-8e9c-4b7c-bb29-0e5162784fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:16.219169 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4310e7-8e9c-4b7c-bb29-0e5162784fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:16.219438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4310e7-8e9c-4b7c-bb29-0e5162784fa5 None admin] PATH_INFO: `/v3/projects/b8e96d6e59f84824b35d74726a4d7135` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:16.220560 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4310e7-8e9c-4b7c-bb29-0e5162784fa5 None admin] RBAC: Authorizing `identity:get_project(project_id=b8e96d6e59f84824b35d74726a4d7135)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:16.221983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4310e7-8e9c-4b7c-bb29-0e5162784fa5 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:16.223628 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 554/1122] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:53:16 2022] GET /identity/v3/projects/b8e96d6e59f84824b35d74726a4d7135 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:16.229501 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7c680-fd3e-4e12-bce8-9113e882d7b7 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:16.298214 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:16.328501 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7c680-fd3e-4e12-bce8-9113e882d7b7 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:16.362064 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d397c53-64ec-4acf-8b45-bc529faed5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:16.362712 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d397c53-64ec-4acf-8b45-bc529faed5e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:16.362990 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d397c53-64ec-4acf-8b45-bc529faed5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:16.363258 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d397c53-64ec-4acf-8b45-bc529faed5e5 None admin] PATH_INFO: `/v3/projects/b8e96d6e59f84824b35d74726a4d7135/users/32b5d85296d747b19042816650478bb2/roles/89a3bc954b0a45bfa40a16db0ad2e51c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:16.365401 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d397c53-64ec-4acf-8b45-bc529faed5e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b8e96d6e59f84824b35d74726a4d7135, user_id=32b5d85296d747b19042816650478bb2, role_id=89a3bc954b0a45bfa40a16db0ad2e51c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:16.366823 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d397c53-64ec-4acf-8b45-bc529faed5e5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:16.379154 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 569/1123] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 00:53:16 2022] PUT /identity/v3/projects/b8e96d6e59f84824b35d74726a4d7135/users/32b5d85296d747b19042816650478bb2/roles/89a3bc954b0a45bfa40a16db0ad2e51c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:53:18.546980 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9baccf-307f-4c69-adea-6bfcae7f591d None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:18.547579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9baccf-307f-4c69-adea-6bfcae7f591d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:18.547579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9baccf-307f-4c69-adea-6bfcae7f591d None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:18.548859 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 555/1124] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:53:18.555275 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad55e31-3765-4e17-8391-974016486a70 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:18.555740 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad55e31-3765-4e17-8391-974016486a70 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:18.555853 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad55e31-3765-4e17-8391-974016486a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:18.577529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-5ad55e31-3765-4e17-8391-974016486a70 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:53:18.578745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5ad55e31-3765-4e17-8391-974016486a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:53:18.653331 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 570/1125] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:53:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:53:18.932668 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9baccf-307f-4c69-adea-6bfcae7f591d None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:19.034040 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9baccf-307f-4c69-adea-6bfcae7f591d None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:19.050824 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:19.068866 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d92f23b-10c4-4b69-9723-7fc2a887b87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:19.069493 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92f23b-10c4-4b69-9723-7fc2a887b87a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:19.069771 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92f23b-10c4-4b69-9723-7fc2a887b87a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:19.070042 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92f23b-10c4-4b69-9723-7fc2a887b87a None admin] PATH_INFO: `/v3/roles/89a3bc954b0a45bfa40a16db0ad2e51c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:19.071397 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d92f23b-10c4-4b69-9723-7fc2a887b87a None admin] RBAC: Authorizing `identity:get_role(role_id=89a3bc954b0a45bfa40a16db0ad2e51c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:19.072801 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d92f23b-10c4-4b69-9723-7fc2a887b87a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:19.074114 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 556/1126] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:53:18 2022] GET /identity/v3/roles/89a3bc954b0a45bfa40a16db0ad2e51c => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:19.080097 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad55e31-3765-4e17-8391-974016486a70 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:19.137275 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:19.178433 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad55e31-3765-4e17-8391-974016486a70 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:19.211325 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c0809-a927-435c-824d-27ea721bad4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:19.211902 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c0809-a927-435c-824d-27ea721bad4a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:19.212180 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c0809-a927-435c-824d-27ea721bad4a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:19.212451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c0809-a927-435c-824d-27ea721bad4a None admin] PATH_INFO: `/v3/users/32b5d85296d747b19042816650478bb2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:19.213547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7c0809-a927-435c-824d-27ea721bad4a None admin] RBAC: Authorizing `identity:get_user(user_id=32b5d85296d747b19042816650478bb2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:19.214972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7c0809-a927-435c-824d-27ea721bad4a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:19.217426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 571/1127] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:53:19 2022] GET /identity/v3/users/32b5d85296d747b19042816650478bb2 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:19.223200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d92f23b-10c4-4b69-9723-7fc2a887b87a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:19.320813 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d92f23b-10c4-4b69-9723-7fc2a887b87a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:19.353962 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b1132-e7eb-4c25-a7f1-e8e9350cace7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:19.354557 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b1132-e7eb-4c25-a7f1-e8e9350cace7 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:19.354832 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b1132-e7eb-4c25-a7f1-e8e9350cace7 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:19.355100 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b1132-e7eb-4c25-a7f1-e8e9350cace7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:19.405841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:19.421590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b1132-e7eb-4c25-a7f1-e8e9350cace7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:19.423027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b1132-e7eb-4c25-a7f1-e8e9350cace7 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:19.519099 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 557/1128] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:53:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 00:53:19.524974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c0809-a927-435c-824d-27ea721bad4a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:19.624545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7c0809-a927-435c-824d-27ea721bad4a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:19.649639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:19.658357 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1049d-4704-4e97-9d62-6b7d67e37ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:19.658972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1049d-4704-4e97-9d62-6b7d67e37ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:19.659257 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1049d-4704-4e97-9d62-6b7d67e37ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:19.659533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1049d-4704-4e97-9d62-6b7d67e37ea6 None admin] PATH_INFO: `/v3/projects/b8e96d6e59f84824b35d74726a4d7135` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:19.660682 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1049d-4704-4e97-9d62-6b7d67e37ea6 None admin] RBAC: Authorizing `identity:get_project(project_id=b8e96d6e59f84824b35d74726a4d7135)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:19.662116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1049d-4704-4e97-9d62-6b7d67e37ea6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:19.663768 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 572/1129] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:53:19 2022] GET /identity/v3/projects/b8e96d6e59f84824b35d74726a4d7135 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:19.670731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b1132-e7eb-4c25-a7f1-e8e9350cace7 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:19.752081 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:19.767844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b1132-e7eb-4c25-a7f1-e8e9350cace7 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:19.801450 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a7284-25be-4514-b363-2ef1143211c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:19.802041 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a7284-25be-4514-b363-2ef1143211c8 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:19.802320 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a7284-25be-4514-b363-2ef1143211c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:19.802592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a7284-25be-4514-b363-2ef1143211c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:19.803298 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941a7284-25be-4514-b363-2ef1143211c8 None admin] RBAC: Adding query filter params (role.id=89a3bc954b0a45bfa40a16db0ad2e51c, scope.project.id=b8e96d6e59f84824b35d74726a4d7135, user.id=32b5d85296d747b19042816650478bb2) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:53:19.803628 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941a7284-25be-4514-b363-2ef1143211c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:19.805051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941a7284-25be-4514-b363-2ef1143211c8 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:19.810592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 558/1130] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:53:19 2022] GET /identity/v3/role_assignments?user.id=32b5d85296d747b19042816650478bb2&scope.project.id=b8e96d6e59f84824b35d74726a4d7135&role.id=89a3bc954b0a45bfa40a16db0ad2e51c => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:21.965108 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb2fe5-62b5-4987-98a6-427ded6a926e None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:21.965764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb2fe5-62b5-4987-98a6-427ded6a926e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:21.965764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb2fe5-62b5-4987-98a6-427ded6a926e None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:21.967174 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 573/1131] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:53:21.972340 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae373eed-08fc-456a-8259-df8f82870fd7 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:21.972785 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae373eed-08fc-456a-8259-df8f82870fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:21.972882 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae373eed-08fc-456a-8259-df8f82870fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:21.990538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-ae373eed-08fc-456a-8259-df8f82870fd7 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:53:21.991736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ae373eed-08fc-456a-8259-df8f82870fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:53:22.055460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 559/1132] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:53:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:53:22.320773 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb2fe5-62b5-4987-98a6-427ded6a926e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:22.425837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb2fe5-62b5-4987-98a6-427ded6a926e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:22.459283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c74824-e4b5-45c7-aee3-13cd1d158bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:22.459931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c74824-e4b5-45c7-aee3-13cd1d158bce None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:22.460211 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c74824-e4b5-45c7-aee3-13cd1d158bce None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:22.460484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c74824-e4b5-45c7-aee3-13cd1d158bce None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:22.461579 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c74824-e4b5-45c7-aee3-13cd1d158bce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:22.463006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c74824-e4b5-45c7-aee3-13cd1d158bce None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:22.464586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 574/1133] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:53:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:22.471009 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae373eed-08fc-456a-8259-df8f82870fd7 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:22.548204 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:22.569818 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae373eed-08fc-456a-8259-df8f82870fd7 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:22.602656 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac059e-187b-4f09-a53b-2f3f15ed2452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:22.603191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac059e-187b-4f09-a53b-2f3f15ed2452 None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:22.603466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac059e-187b-4f09-a53b-2f3f15ed2452 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:22.603762 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac059e-187b-4f09-a53b-2f3f15ed2452 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:22.604402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac059e-187b-4f09-a53b-2f3f15ed2452 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:22.605814 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac059e-187b-4f09-a53b-2f3f15ed2452 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:22.632762 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 560/1134] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 00:53:22 2022] POST /identity/v3/projects => generated 311 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:53:24.802396 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1401e-6afe-4f51-ae01-b8ee1eeaa421 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:24.802902 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1401e-6afe-4f51-ae01-b8ee1eeaa421 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:24.802902 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1401e-6afe-4f51-ae01-b8ee1eeaa421 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:24.804236 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 575/1135] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:53:24.809855 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf037f12-e3a8-4a71-badb-6ca7fcb16ed8 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:24.810236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf037f12-e3a8-4a71-badb-6ca7fcb16ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:24.810413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf037f12-e3a8-4a71-badb-6ca7fcb16ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:24.828872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-cf037f12-e3a8-4a71-badb-6ca7fcb16ed8 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:53:24.830040 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-cf037f12-e3a8-4a71-badb-6ca7fcb16ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:53:24.895023 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 561/1136] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:53:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:53:25.170837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a1401e-6afe-4f51-ae01-b8ee1eeaa421 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:25.234637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:25.277280 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a1401e-6afe-4f51-ae01-b8ee1eeaa421 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:25.310942 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b40b0c-9bb6-423c-9465-ba8f21bafd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:25.311576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b40b0c-9bb6-423c-9465-ba8f21bafd4d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:25.311875 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b40b0c-9bb6-423c-9465-ba8f21bafd4d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:25.312149 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b40b0c-9bb6-423c-9465-ba8f21bafd4d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:25.313249 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b40b0c-9bb6-423c-9465-ba8f21bafd4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:25.314682 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b40b0c-9bb6-423c-9465-ba8f21bafd4d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:25.316263 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 576/1137] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:53:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:25.321977 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf037f12-e3a8-4a71-badb-6ca7fcb16ed8 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:25.379113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:25.419708 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf037f12-e3a8-4a71-badb-6ca7fcb16ed8 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:25.452851 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3918f2f-279f-48d2-afa2-6155d97122fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:25.453388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3918f2f-279f-48d2-afa2-6155d97122fe None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:25.453669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3918f2f-279f-48d2-afa2-6155d97122fe None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:25.453942 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3918f2f-279f-48d2-afa2-6155d97122fe None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:25.454594 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3918f2f-279f-48d2-afa2-6155d97122fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:25.456066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3918f2f-279f-48d2-afa2-6155d97122fe None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:25.477783 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 562/1138] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 00:53:25 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:53:27.638837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ab8b4-988b-4edd-9381-78435535ba91 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:27.639360 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ab8b4-988b-4edd-9381-78435535ba91 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:27.639360 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ab8b4-988b-4edd-9381-78435535ba91 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:27.641016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 577/1139] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:53:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:53:27.646279 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f5182-3bca-4d5b-a85f-37a17f98397f None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:27.646713 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f5182-3bca-4d5b-a85f-37a17f98397f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:27.646831 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f5182-3bca-4d5b-a85f-37a17f98397f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:27.665475 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-7c0f5182-3bca-4d5b-a85f-37a17f98397f None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:53:27.666667 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7c0f5182-3bca-4d5b-a85f-37a17f98397f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:53:27.731591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 563/1140] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:53:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:53:28.002025 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ab8b4-988b-4edd-9381-78435535ba91 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:28.100924 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ab8b4-988b-4edd-9381-78435535ba91 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:28.129427 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:28.134574 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898b2b1-f1d9-44b4-8700-f6c6da36f55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:28.135197 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898b2b1-f1d9-44b4-8700-f6c6da36f55d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:28.135479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898b2b1-f1d9-44b4-8700-f6c6da36f55d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:28.135781 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898b2b1-f1d9-44b4-8700-f6c6da36f55d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:28.146549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0898b2b1-f1d9-44b4-8700-f6c6da36f55d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:53:28.151345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-0898b2b1-f1d9-44b4-8700-f6c6da36f55d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:53:28.151345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:53:28.151345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:53:28.151345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:53:28.151345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:53:28.151345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:53:28.151345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:53:28.151345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:53:28.151345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:53:28.151345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:53:28.151345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:53:28.151345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:53:28.151345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:53:28.151345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:53:28.151345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:53:28.151345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:53:28.152431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:53:28.152431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:53:28.152431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:53:28.152431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:53:28.152431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:53:28.152431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:53:28.152431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:53:28.152431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:53:28.152431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:53:28.152431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:53:28.152431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:53:28.152431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:53:28.152431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:53:28.152431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:53:28.152431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:53:28.152431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:53:28.153810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:53:28.153810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:53:28.153810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:53:28.153810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:53:28.153810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:53:28.153810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:53:28.153810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:53:28.153810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:53:28.153810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:53:28.153810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:53:28.153810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:53:28.153810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:53:28.153810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 578/1141] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:53:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:53:28.158595 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f5182-3bca-4d5b-a85f-37a17f98397f None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:28.204853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:28.257856 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f5182-3bca-4d5b-a85f-37a17f98397f None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:28.291592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d4608-261b-41f3-ad60-0c3247cc5ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:28.292193 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d4608-261b-41f3-ad60-0c3247cc5ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:28.292473 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d4608-261b-41f3-ad60-0c3247cc5ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:28.292747 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d4608-261b-41f3-ad60-0c3247cc5ca6 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:28.293929 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d4608-261b-41f3-ad60-0c3247cc5ca6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:53:28.294262 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d4608-261b-41f3-ad60-0c3247cc5ca6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:28.295657 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d4608-261b-41f3-ad60-0c3247cc5ca6 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:28.303402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 564/1142] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:53:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:28.309967 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898b2b1-f1d9-44b4-8700-f6c6da36f55d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:28.410487 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898b2b1-f1d9-44b4-8700-f6c6da36f55d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:28.443539 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4201fa-b9b9-4dbf-bb57-b2aee5f843cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:28.444185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4201fa-b9b9-4dbf-bb57-b2aee5f843cb None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:28.444466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4201fa-b9b9-4dbf-bb57-b2aee5f843cb None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:28.444738 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4201fa-b9b9-4dbf-bb57-b2aee5f843cb None admin] PATH_INFO: `/v3/users/3ea9eb0f418948518adad0489b269137` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:28.458061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4201fa-b9b9-4dbf-bb57-b2aee5f843cb None admin] RBAC: Authorizing `identity:get_user(user_id=3ea9eb0f418948518adad0489b269137)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:28.459502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4201fa-b9b9-4dbf-bb57-b2aee5f843cb None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:28.461111 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 579/1143] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:53:28 2022] GET /identity/v3/users/3ea9eb0f418948518adad0489b269137 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:28.467474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d4608-261b-41f3-ad60-0c3247cc5ca6 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:28.564626 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d4608-261b-41f3-ad60-0c3247cc5ca6 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:28.573224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:28.598729 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e91de9-c3ad-4088-ae5b-3d7b6fa99818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:28.599350 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e91de9-c3ad-4088-ae5b-3d7b6fa99818 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:28.599658 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e91de9-c3ad-4088-ae5b-3d7b6fa99818 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:28.599935 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e91de9-c3ad-4088-ae5b-3d7b6fa99818 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:28.664967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e91de9-c3ad-4088-ae5b-3d7b6fa99818 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:28.666401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e91de9-c3ad-4088-ae5b-3d7b6fa99818 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:28.761278 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 565/1144] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:53:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 00:53:28.768261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4201fa-b9b9-4dbf-bb57-b2aee5f843cb None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:28.817233 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:28.865794 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4201fa-b9b9-4dbf-bb57-b2aee5f843cb None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:28.899727 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a04476-32d4-41a3-bb92-15ed1d9642d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:28.900353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a04476-32d4-41a3-bb92-15ed1d9642d0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:28.900637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a04476-32d4-41a3-bb92-15ed1d9642d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:28.900911 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a04476-32d4-41a3-bb92-15ed1d9642d0 None admin] PATH_INFO: `/v3/projects/11f3fdf75cb444da93e39338ca51fd19` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:28.902031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a04476-32d4-41a3-bb92-15ed1d9642d0 None admin] RBAC: Authorizing `identity:get_project(project_id=11f3fdf75cb444da93e39338ca51fd19)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:28.903490 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a04476-32d4-41a3-bb92-15ed1d9642d0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:28.905122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 580/1145] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:53:28 2022] GET /identity/v3/projects/11f3fdf75cb444da93e39338ca51fd19 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:28.913160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e91de9-c3ad-4088-ae5b-3d7b6fa99818 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:28.923339 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:29.011215 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e91de9-c3ad-4088-ae5b-3d7b6fa99818 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:29.045418 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb57f9-87f2-4bc0-bea0-48f9746c9b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:29.046015 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb57f9-87f2-4bc0-bea0-48f9746c9b7d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:29.046296 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb57f9-87f2-4bc0-bea0-48f9746c9b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:29.046568 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb57f9-87f2-4bc0-bea0-48f9746c9b7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:29.047275 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb57f9-87f2-4bc0-bea0-48f9746c9b7d None admin] RBAC: Adding query filter params (role.id=ada268ea63354a0297070369dfe3c6cc, scope.project.id=11f3fdf75cb444da93e39338ca51fd19, user.id=3ea9eb0f418948518adad0489b269137) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:53:29.047609 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb57f9-87f2-4bc0-bea0-48f9746c9b7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:29.049037 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb57f9-87f2-4bc0-bea0-48f9746c9b7d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:29.054180 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 566/1146] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:53:28 2022] GET /identity/v3/role_assignments?user.id=3ea9eb0f418948518adad0489b269137&scope.project.id=11f3fdf75cb444da93e39338ca51fd19&role.id=ada268ea63354a0297070369dfe3c6cc => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:31.170793 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83b7fa-1f74-44f8-b1a7-bc45e46cf2f7 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:31.171355 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83b7fa-1f74-44f8-b1a7-bc45e46cf2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:31.171355 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83b7fa-1f74-44f8-b1a7-bc45e46cf2f7 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:31.172728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 581/1147] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:53:31.177862 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff970e5-c00e-48ef-beec-196e4c7c51b3 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:31.178308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff970e5-c00e-48ef-beec-196e4c7c51b3 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:31.178401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff970e5-c00e-48ef-beec-196e4c7c51b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:31.196693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-0ff970e5-c00e-48ef-beec-196e4c7c51b3 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:53:31.197876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0ff970e5-c00e-48ef-beec-196e4c7c51b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:53:31.263244 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 567/1148] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:53:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:53:31.533888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83b7fa-1f74-44f8-b1a7-bc45e46cf2f7 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:31.633866 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83b7fa-1f74-44f8-b1a7-bc45e46cf2f7 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:31.651242 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:31.668513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564ec3a-fcd9-497c-ab61-b879188c9348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:31.669204 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564ec3a-fcd9-497c-ab61-b879188c9348 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:31.669485 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564ec3a-fcd9-497c-ab61-b879188c9348 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:31.669758 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564ec3a-fcd9-497c-ab61-b879188c9348 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:31.679520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564ec3a-fcd9-497c-ab61-b879188c9348 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:53:31.684936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-3564ec3a-fcd9-497c-ab61-b879188c9348 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:53:31.684936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:53:31.684936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:53:31.684936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:53:31.684936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:53:31.684936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:53:31.684936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:53:31.684936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:53:31.684936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:53:31.684936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:53:31.684936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:53:31.684936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:53:31.684936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:53:31.684936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:53:31.684936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:53:31.684936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:53:31.685977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:53:31.685977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:53:31.685977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:53:31.685977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:53:31.685977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:53:31.685977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:53:31.685977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:53:31.685977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:53:31.685977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:53:31.685977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:53:31.685977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:53:31.685977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:53:31.685977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:53:31.685977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:53:31.685977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:53:31.685977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:53:31.687073 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:53:31.687073 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:53:31.687073 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:53:31.687073 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:53:31.687073 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:53:31.687073 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:53:31.687073 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:53:31.687073 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:53:31.687073 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:53:31.687073 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:53:31.687073 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:53:31.687073 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:53:31.687073 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 582/1149] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:53:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:53:31.691999 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff970e5-c00e-48ef-beec-196e4c7c51b3 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:31.694201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:31.788796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff970e5-c00e-48ef-beec-196e4c7c51b3 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:31.822150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a742d3de-910a-4ce4-998f-6b95a5feda54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:31.822745 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742d3de-910a-4ce4-998f-6b95a5feda54 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:31.823025 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742d3de-910a-4ce4-998f-6b95a5feda54 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:31.823298 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742d3de-910a-4ce4-998f-6b95a5feda54 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:31.823980 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a742d3de-910a-4ce4-998f-6b95a5feda54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:53:31.824884 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a742d3de-910a-4ce4-998f-6b95a5feda54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:31.826229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a742d3de-910a-4ce4-998f-6b95a5feda54 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:31.834000 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 568/1150] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:53:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:31.839771 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564ec3a-fcd9-497c-ab61-b879188c9348 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:31.939255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564ec3a-fcd9-497c-ab61-b879188c9348 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:31.972330 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e206d-a6f0-430a-8592-3a745e54ce85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:31.972949 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e206d-a6f0-430a-8592-3a745e54ce85 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:31.973231 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e206d-a6f0-430a-8592-3a745e54ce85 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:31.973507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e206d-a6f0-430a-8592-3a745e54ce85 None admin] PATH_INFO: `/v3/users/3ea9eb0f418948518adad0489b269137` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:31.974620 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e206d-a6f0-430a-8592-3a745e54ce85 None admin] RBAC: Authorizing `identity:get_user(user_id=3ea9eb0f418948518adad0489b269137)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:31.976082 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e206d-a6f0-430a-8592-3a745e54ce85 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:31.977639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 583/1151] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:53:31 2022] GET /identity/v3/users/3ea9eb0f418948518adad0489b269137 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:31.983432 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a742d3de-910a-4ce4-998f-6b95a5feda54 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:32.044840 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:32.081708 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a742d3de-910a-4ce4-998f-6b95a5feda54 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:32.114371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0cc38-003e-4777-ad56-87b9f4c7439e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:32.114992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0cc38-003e-4777-ad56-87b9f4c7439e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:32.115276 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0cc38-003e-4777-ad56-87b9f4c7439e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:32.115583 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0cc38-003e-4777-ad56-87b9f4c7439e None admin] PATH_INFO: `/v3/projects/11f3fdf75cb444da93e39338ca51fd19` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:32.116714 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0cc38-003e-4777-ad56-87b9f4c7439e None admin] RBAC: Authorizing `identity:get_project(project_id=11f3fdf75cb444da93e39338ca51fd19)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:32.118146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0cc38-003e-4777-ad56-87b9f4c7439e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:32.119844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 569/1152] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:53:31 2022] GET /identity/v3/projects/11f3fdf75cb444da93e39338ca51fd19 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:32.126264 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e206d-a6f0-430a-8592-3a745e54ce85 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:32.172355 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:32.223886 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e206d-a6f0-430a-8592-3a745e54ce85 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:32.257337 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f38da-30c0-4e8b-8032-fe9c876763ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:32.257987 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f38da-30c0-4e8b-8032-fe9c876763ee None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:32.258271 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f38da-30c0-4e8b-8032-fe9c876763ee None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:32.258551 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f38da-30c0-4e8b-8032-fe9c876763ee None admin] PATH_INFO: `/v3/projects/11f3fdf75cb444da93e39338ca51fd19/users/3ea9eb0f418948518adad0489b269137/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:32.260469 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f38da-30c0-4e8b-8032-fe9c876763ee None admin] RBAC: Authorizing `identity:create_grant(project_id=11f3fdf75cb444da93e39338ca51fd19, user_id=3ea9eb0f418948518adad0489b269137, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:32.262035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f38da-30c0-4e8b-8032-fe9c876763ee None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:32.275402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 584/1153] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 00:53:32 2022] PUT /identity/v3/projects/11f3fdf75cb444da93e39338ca51fd19/users/3ea9eb0f418948518adad0489b269137/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:53:34.446130 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18502f-1125-4920-9384-14d7ecfea708 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:34.446686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18502f-1125-4920-9384-14d7ecfea708 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:34.446686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18502f-1125-4920-9384-14d7ecfea708 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:34.448051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 570/1154] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:53:34.453494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a056f8-5294-44ca-b45a-e87cb221295d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:34.453880 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a056f8-5294-44ca-b45a-e87cb221295d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:34.454042 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a056f8-5294-44ca-b45a-e87cb221295d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:34.471927 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-85a056f8-5294-44ca-b45a-e87cb221295d None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:53:34.473106 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-85a056f8-5294-44ca-b45a-e87cb221295d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:53:34.539605 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 585/1155] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:53:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:53:34.815972 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18502f-1125-4920-9384-14d7ecfea708 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:34.915230 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18502f-1125-4920-9384-14d7ecfea708 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:34.943643 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:34.948959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0230a3-5330-4643-8e9b-53c74f3154ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:34.950243 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0230a3-5330-4643-8e9b-53c74f3154ce None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:34.950697 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0230a3-5330-4643-8e9b-53c74f3154ce None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:34.950979 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0230a3-5330-4643-8e9b-53c74f3154ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:34.961390 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0230a3-5330-4643-8e9b-53c74f3154ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:53:34.966413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-7f0230a3-5330-4643-8e9b-53c74f3154ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:53:34.966413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:53:34.966413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:53:34.966413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:53:34.966413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:53:34.966413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:53:34.966413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:53:34.966413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:53:34.966413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:53:34.966413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:53:34.966413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:53:34.966413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:53:34.966413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:53:34.966413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:53:34.966413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:53:34.966413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:53:34.967584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:53:34.967584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:53:34.967584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:53:34.967584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:53:34.967584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:53:34.967584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:53:34.967584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:53:34.967584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:53:34.967584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:53:34.967584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:53:34.967584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:53:34.967584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:53:34.967584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:53:34.967584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:53:34.967584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:53:34.967584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:53:34.968906 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:53:34.968906 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:53:34.968906 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:53:34.968906 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:53:34.968906 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:53:34.968906 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:53:34.968906 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:53:34.968906 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:53:34.968906 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:53:34.968906 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:53:34.968906 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:53:34.968906 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:53:34.968906 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 571/1156] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:53:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:53:34.973574 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a056f8-5294-44ca-b45a-e87cb221295d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:35.007965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:35.072479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a056f8-5294-44ca-b45a-e87cb221295d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:35.106444 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fd218-8005-4aac-9df3-5f310f37e745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:35.107068 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fd218-8005-4aac-9df3-5f310f37e745 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:35.107352 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fd218-8005-4aac-9df3-5f310f37e745 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:35.107651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fd218-8005-4aac-9df3-5f310f37e745 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:35.108311 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fd218-8005-4aac-9df3-5f310f37e745 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:53:35.108615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fd218-8005-4aac-9df3-5f310f37e745 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:35.109983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fd218-8005-4aac-9df3-5f310f37e745 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:35.117821 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 586/1157] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:53:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:35.124785 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0230a3-5330-4643-8e9b-53c74f3154ce None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:35.221481 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0230a3-5330-4643-8e9b-53c74f3154ce None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:35.254584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8311944-8948-418b-9105-bbc64a44181f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:35.255197 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8311944-8948-418b-9105-bbc64a44181f None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:35.255476 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8311944-8948-418b-9105-bbc64a44181f None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:35.255776 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8311944-8948-418b-9105-bbc64a44181f None admin] PATH_INFO: `/v3/users/3ea9eb0f418948518adad0489b269137` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:35.256869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8311944-8948-418b-9105-bbc64a44181f None admin] RBAC: Authorizing `identity:get_user(user_id=3ea9eb0f418948518adad0489b269137)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:35.258291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8311944-8948-418b-9105-bbc64a44181f None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:35.259864 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 572/1158] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:53:35 2022] GET /identity/v3/users/3ea9eb0f418948518adad0489b269137 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:35.265863 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fd218-8005-4aac-9df3-5f310f37e745 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:35.359943 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:35.364009 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fd218-8005-4aac-9df3-5f310f37e745 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:35.397671 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3f669-041a-4671-9166-60b0a76b6b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:35.398274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3f669-041a-4671-9166-60b0a76b6b19 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:35.398560 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3f669-041a-4671-9166-60b0a76b6b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:35.398833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3f669-041a-4671-9166-60b0a76b6b19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:35.464383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e3f669-041a-4671-9166-60b0a76b6b19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:35.466366 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e3f669-041a-4671-9166-60b0a76b6b19 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:35.558889 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:35.563480 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 587/1159] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:53:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 00:53:35.569473 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8311944-8948-418b-9105-bbc64a44181f None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:35.627266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:35.667175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8311944-8948-418b-9105-bbc64a44181f None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:35.700026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c8036-af98-4fc3-954b-8ffc3a79a184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:35.700642 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c8036-af98-4fc3-954b-8ffc3a79a184 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:35.700924 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c8036-af98-4fc3-954b-8ffc3a79a184 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:35.701198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c8036-af98-4fc3-954b-8ffc3a79a184 None admin] PATH_INFO: `/v3/projects/11f3fdf75cb444da93e39338ca51fd19` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:35.702292 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c8036-af98-4fc3-954b-8ffc3a79a184 None admin] RBAC: Authorizing `identity:get_project(project_id=11f3fdf75cb444da93e39338ca51fd19)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:35.703762 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c8036-af98-4fc3-954b-8ffc3a79a184 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:35.705365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 573/1160] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:53:35 2022] GET /identity/v3/projects/11f3fdf75cb444da93e39338ca51fd19 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:35.711519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3f669-041a-4671-9166-60b0a76b6b19 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:35.807719 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3f669-041a-4671-9166-60b0a76b6b19 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:35.840576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb78af-4e5b-4274-90d8-83fd69e6e912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:35.841166 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb78af-4e5b-4274-90d8-83fd69e6e912 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:35.841444 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb78af-4e5b-4274-90d8-83fd69e6e912 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:35.841715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb78af-4e5b-4274-90d8-83fd69e6e912 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:35.842418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb78af-4e5b-4274-90d8-83fd69e6e912 None admin] RBAC: Adding query filter params (role.id=ada268ea63354a0297070369dfe3c6cc, scope.project.id=11f3fdf75cb444da93e39338ca51fd19, user.id=3ea9eb0f418948518adad0489b269137) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:53:35.842723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb78af-4e5b-4274-90d8-83fd69e6e912 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:35.844160 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb78af-4e5b-4274-90d8-83fd69e6e912 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:35.849695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 588/1161] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:53:35 2022] GET /identity/v3/role_assignments?user.id=3ea9eb0f418948518adad0489b269137&scope.project.id=11f3fdf75cb444da93e39338ca51fd19&role.id=ada268ea63354a0297070369dfe3c6cc => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:38.024212 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772099f5-ec7f-4f28-95e3-4618308abd8f None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:38.024763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772099f5-ec7f-4f28-95e3-4618308abd8f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:38.024763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772099f5-ec7f-4f28-95e3-4618308abd8f None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:38.026080 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 574/1162] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:53:38.032095 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7fadb-d004-4e97-ad0d-3538b13cb4d9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:38.032589 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7fadb-d004-4e97-ad0d-3538b13cb4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:38.032684 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7fadb-d004-4e97-ad0d-3538b13cb4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:38.051063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-96b7fadb-d004-4e97-ad0d-3538b13cb4d9 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:53:38.052281 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-96b7fadb-d004-4e97-ad0d-3538b13cb4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:53:38.107459 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:38.117916 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 589/1163] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:53:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 00:53:38.389180 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772099f5-ec7f-4f28-95e3-4618308abd8f None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:38.495235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772099f5-ec7f-4f28-95e3-4618308abd8f None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:38.522038 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:38.530048 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3a3e0-f4a1-409f-871d-dcfe39b4be4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:38.530645 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3a3e0-f4a1-409f-871d-dcfe39b4be4a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:38.530924 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3a3e0-f4a1-409f-871d-dcfe39b4be4a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:38.531196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3a3e0-f4a1-409f-871d-dcfe39b4be4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:38.597007 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3a3e0-f4a1-409f-871d-dcfe39b4be4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:38.598469 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3a3e0-f4a1-409f-871d-dcfe39b4be4a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:38.693191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 575/1164] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:53:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 00:53:38.699219 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7fadb-d004-4e97-ad0d-3538b13cb4d9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:38.795493 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7fadb-d004-4e97-ad0d-3538b13cb4d9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:38.828429 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f5bef0-014a-4e1b-a28d-92a1dc3d3147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:38.829047 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5bef0-014a-4e1b-a28d-92a1dc3d3147 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:38.829330 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5bef0-014a-4e1b-a28d-92a1dc3d3147 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:38.829603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5bef0-014a-4e1b-a28d-92a1dc3d3147 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:38.836045 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f5bef0-014a-4e1b-a28d-92a1dc3d3147 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:38.837488 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f5bef0-014a-4e1b-a28d-92a1dc3d3147 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:38.848885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-f1f5bef0-014a-4e1b-a28d-92a1dc3d3147 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 14 00:53:38.848885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:53:38.848885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 14 00:53:38.848885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 14 00:53:38.848885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:53:38.848885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:53:38.848885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:53:38.848885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:53:38.848885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 14 00:53:38.848885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:53:38.848885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:53:38.848885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:53:38.848885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:53:38.848885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:53:38.848885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:53:38.848885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:53:38.848885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:53:38.850129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:53:38.850129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:53:38.850129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:53:38.850129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:53:38.850129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:53:38.850129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:53:38.850129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:53:38.850129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:53:38.850129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:53:38.850129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:53:38.850129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:53:38.850129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:53:38.850129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:53:38.850129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:53:38.850129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:53:38.850129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:53:38.851202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:53:38.851202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:53:38.851202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:53:38.851202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:53:38.851202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:53:38.851202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:53:38.851202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:53:38.851202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:53:38.851202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:53:38.851202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:53:38.851202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 14 00:53:38.851202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:53:38.851202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 14 00:53:38.851202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:53:38.851202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 590/1165] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Sep 14 00:53:38 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:53:38.856846 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3a3e0-f4a1-409f-871d-dcfe39b4be4a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:38.884099 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:38.954735 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3a3e0-f4a1-409f-871d-dcfe39b4be4a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:38.988159 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d2dc7-0bbc-4681-b69b-826616de6a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:38.988748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d2dc7-0bbc-4681-b69b-826616de6a9a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:38.989024 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d2dc7-0bbc-4681-b69b-826616de6a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:38.989294 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d2dc7-0bbc-4681-b69b-826616de6a9a None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:38.989921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d2dc7-0bbc-4681-b69b-826616de6a9a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:53:38.990219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d2dc7-0bbc-4681-b69b-826616de6a9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:38.991594 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d2dc7-0bbc-4681-b69b-826616de6a9a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:38.999354 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 576/1166] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Sep 14 00:53:38 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:41.132803 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f34444-6a21-4f65-9ff8-ce9323837377 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:41.133360 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f34444-6a21-4f65-9ff8-ce9323837377 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:41.133360 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f34444-6a21-4f65-9ff8-ce9323837377 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:41.134675 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 591/1167] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:53:41.140862 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb540f8-ac11-4fa8-8a4e-31871314dd24 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:41.141254 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb540f8-ac11-4fa8-8a4e-31871314dd24 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:41.141410 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb540f8-ac11-4fa8-8a4e-31871314dd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:41.159869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-1bb540f8-ac11-4fa8-8a4e-31871314dd24 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:53:41.161039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1bb540f8-ac11-4fa8-8a4e-31871314dd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:53:41.225528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 577/1168] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:53:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:53:41.494310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f34444-6a21-4f65-9ff8-ce9323837377 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:41.548126 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:41.595637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f34444-6a21-4f65-9ff8-ce9323837377 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:41.628837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09493292-c229-4bf7-8664-e62618b570d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:41.629436 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09493292-c229-4bf7-8664-e62618b570d5 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:41.629719 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09493292-c229-4bf7-8664-e62618b570d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:41.629993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09493292-c229-4bf7-8664-e62618b570d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:41.630664 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09493292-c229-4bf7-8664-e62618b570d5 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:41.632425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09493292-c229-4bf7-8664-e62618b570d5 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:41.648357 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 592/1169] 10.222.0.42 () {60 vars in 1138 bytes} [Wed Sep 14 00:53:41 2022] POST /identity/v3/domains => generated 267 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:53:43.815469 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e2768-48f9-470d-a054-114ca1f9d652 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:43.816032 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e2768-48f9-470d-a054-114ca1f9d652 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:43.816032 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e2768-48f9-470d-a054-114ca1f9d652 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:43.817330 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 578/1170] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:53:43.822486 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff66a0-5511-4c72-a0e2-c4c938611755 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:43.822870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff66a0-5511-4c72-a0e2-c4c938611755 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:43.823044 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff66a0-5511-4c72-a0e2-c4c938611755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:43.842533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c5ff66a0-5511-4c72-a0e2-c4c938611755 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:53:43.843749 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c5ff66a0-5511-4c72-a0e2-c4c938611755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:53:43.907776 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 593/1171] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:53:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:53:44.178993 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e2768-48f9-470d-a054-114ca1f9d652 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:44.203945 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:44.281677 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e2768-48f9-470d-a054-114ca1f9d652 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:44.314251 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127e226-0db2-41f3-8f5e-050d0e0e6baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:44.314876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127e226-0db2-41f3-8f5e-050d0e0e6baa None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:44.315188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127e226-0db2-41f3-8f5e-050d0e0e6baa None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:44.315465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127e226-0db2-41f3-8f5e-050d0e0e6baa None admin] PATH_INFO: `/v3/domains/22ca95d66fad43028f6c383a524b76c5` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:44.326373 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127e226-0db2-41f3-8f5e-050d0e0e6baa None admin] RBAC: Authorizing `identity:get_domain(domain_id=22ca95d66fad43028f6c383a524b76c5)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:44.327860 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127e226-0db2-41f3-8f5e-050d0e0e6baa None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:44.329438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 579/1172] 10.222.0.42 () {56 vars in 1183 bytes} [Wed Sep 14 00:53:44 2022] GET /identity/v3/domains/22ca95d66fad43028f6c383a524b76c5 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:44.335331 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff66a0-5511-4c72-a0e2-c4c938611755 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:44.376912 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:44.433803 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff66a0-5511-4c72-a0e2-c4c938611755 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:44.467821 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb79b3-e96b-40a8-9151-27c616edf25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:44.468427 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb79b3-e96b-40a8-9151-27c616edf25b None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:44.468704 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb79b3-e96b-40a8-9151-27c616edf25b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:44.468977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb79b3-e96b-40a8-9151-27c616edf25b None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:44.469625 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb79b3-e96b-40a8-9151-27c616edf25b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:44.471049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb79b3-e96b-40a8-9151-27c616edf25b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:44.498014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 594/1173] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 00:53:44 2022] POST /identity/v3/projects => generated 361 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:53:46.675420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d231d-ef19-425f-9f32-813f31166919 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:46.675984 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d231d-ef19-425f-9f32-813f31166919 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:46.675984 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d231d-ef19-425f-9f32-813f31166919 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:46.677338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 580/1174] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:53:46.682632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f2d5f-8330-4505-b8db-53535ebe069b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:46.683525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f2d5f-8330-4505-b8db-53535ebe069b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:46.683833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f2d5f-8330-4505-b8db-53535ebe069b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:46.702262 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-952f2d5f-8330-4505-b8db-53535ebe069b None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:53:46.703438 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-952f2d5f-8330-4505-b8db-53535ebe069b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:53:46.768733 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 595/1175] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:53:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:53:47.043747 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d231d-ef19-425f-9f32-813f31166919 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:47.125107 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:47.144806 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d231d-ef19-425f-9f32-813f31166919 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:47.178004 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3302a9b-397d-43b8-bf14-fd4aad6d08e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:47.178630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3302a9b-397d-43b8-bf14-fd4aad6d08e6 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:47.178915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3302a9b-397d-43b8-bf14-fd4aad6d08e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:47.179192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3302a9b-397d-43b8-bf14-fd4aad6d08e6 None admin] PATH_INFO: `/v3/domains/22ca95d66fad43028f6c383a524b76c5` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:47.180338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3302a9b-397d-43b8-bf14-fd4aad6d08e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=22ca95d66fad43028f6c383a524b76c5)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:47.181777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3302a9b-397d-43b8-bf14-fd4aad6d08e6 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:47.183400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 581/1176] 10.222.0.42 () {56 vars in 1183 bytes} [Wed Sep 14 00:53:47 2022] GET /identity/v3/domains/22ca95d66fad43028f6c383a524b76c5 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:47.189136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f2d5f-8330-4505-b8db-53535ebe069b None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:47.213363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:47.286567 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f2d5f-8330-4505-b8db-53535ebe069b None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:47.319133 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7f8f1-582e-4a30-b6ff-06ed1a0b3edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:47.319688 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7f8f1-582e-4a30-b6ff-06ed1a0b3edb None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:47.319971 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7f8f1-582e-4a30-b6ff-06ed1a0b3edb None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:47.320241 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7f8f1-582e-4a30-b6ff-06ed1a0b3edb None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:47.320894 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7f8f1-582e-4a30-b6ff-06ed1a0b3edb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:47.322314 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7f8f1-582e-4a30-b6ff-06ed1a0b3edb None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:47.343838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 596/1177] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 00:53:47 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 00:53:49.533941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61edaa9-62d8-42b2-b8ec-e4963e8105a9 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:49.534419 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61edaa9-62d8-42b2-b8ec-e4963e8105a9 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:49.534511 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61edaa9-62d8-42b2-b8ec-e4963e8105a9 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:49.535979 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 582/1178] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:53:49.541280 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d1599-a0ad-4f28-a5fc-a2a10d009e6a None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:49.541716 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d1599-a0ad-4f28-a5fc-a2a10d009e6a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:49.541812 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d1599-a0ad-4f28-a5fc-a2a10d009e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:49.560928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-876d1599-a0ad-4f28-a5fc-a2a10d009e6a None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:53:49.562108 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-876d1599-a0ad-4f28-a5fc-a2a10d009e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:53:49.626319 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 597/1179] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:53:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 00:53:49.896302 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61edaa9-62d8-42b2-b8ec-e4963e8105a9 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:49.997829 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61edaa9-62d8-42b2-b8ec-e4963e8105a9 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:50.030748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18733738-4116-4c14-8b8e-d3041fbfdab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:50.031366 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18733738-4116-4c14-8b8e-d3041fbfdab3 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:50.031684 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18733738-4116-4c14-8b8e-d3041fbfdab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:50.031964 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18733738-4116-4c14-8b8e-d3041fbfdab3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:50.042899 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18733738-4116-4c14-8b8e-d3041fbfdab3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:53:50.047825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-18733738-4116-4c14-8b8e-d3041fbfdab3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:53:50.047825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:53:50.047825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:53:50.047825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:53:50.047825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:53:50.047825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:53:50.047825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:53:50.047825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:53:50.047825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:53:50.047825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:53:50.047825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:53:50.047825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:53:50.047825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:53:50.047825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:53:50.047825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:53:50.047825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:53:50.048934 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:53:50.048934 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:53:50.048934 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:53:50.048934 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:53:50.048934 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:53:50.048934 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:53:50.048934 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:53:50.048934 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:53:50.048934 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:53:50.048934 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:53:50.048934 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:53:50.048934 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:53:50.048934 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:53:50.048934 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:53:50.048934 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:53:50.048934 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:53:50.049954 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:53:50.049954 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:53:50.049954 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:53:50.049954 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:53:50.049954 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:53:50.049954 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:53:50.049954 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:53:50.049954 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:53:50.049954 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:53:50.049954 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:53:50.049954 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:53:50.049954 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:53:50.049954 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 583/1180] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:53:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:53:50.054860 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d1599-a0ad-4f28-a5fc-a2a10d009e6a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:50.066385 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:50.154748 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d1599-a0ad-4f28-a5fc-a2a10d009e6a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:50.187043 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe0f36-863b-4f2a-9d30-7f302ba10755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:50.187669 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe0f36-863b-4f2a-9d30-7f302ba10755 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:50.187952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe0f36-863b-4f2a-9d30-7f302ba10755 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:50.188223 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe0f36-863b-4f2a-9d30-7f302ba10755 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:50.188877 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe0f36-863b-4f2a-9d30-7f302ba10755 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:53:50.189180 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe0f36-863b-4f2a-9d30-7f302ba10755 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:50.190537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe0f36-863b-4f2a-9d30-7f302ba10755 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:50.198235 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 598/1181] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:53:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:50.204596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18733738-4116-4c14-8b8e-d3041fbfdab3 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:50.217612 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:50.303072 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18733738-4116-4c14-8b8e-d3041fbfdab3 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:50.336235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e46c83-bf50-459b-b2cf-815672194549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:50.336853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e46c83-bf50-459b-b2cf-815672194549 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:50.337131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e46c83-bf50-459b-b2cf-815672194549 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:50.337402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e46c83-bf50-459b-b2cf-815672194549 None admin] PATH_INFO: `/v3/users/75522e09da8046e9b1be5687def496f7` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:50.350581 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e46c83-bf50-459b-b2cf-815672194549 None admin] RBAC: Authorizing `identity:get_user(user_id=75522e09da8046e9b1be5687def496f7)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:50.352059 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e46c83-bf50-459b-b2cf-815672194549 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:50.353626 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 584/1182] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:53:50 2022] GET /identity/v3/users/75522e09da8046e9b1be5687def496f7 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:50.359747 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe0f36-863b-4f2a-9d30-7f302ba10755 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:50.436402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:50.465417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe0f36-863b-4f2a-9d30-7f302ba10755 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:50.498885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1075ca6a-2151-4c91-8489-8ac47beab26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:50.499491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075ca6a-2151-4c91-8489-8ac47beab26d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:50.499804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075ca6a-2151-4c91-8489-8ac47beab26d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:50.500078 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075ca6a-2151-4c91-8489-8ac47beab26d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:50.565917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1075ca6a-2151-4c91-8489-8ac47beab26d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:50.567358 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1075ca6a-2151-4c91-8489-8ac47beab26d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:50.632087 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:50.662496 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 599/1183] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:53:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 14 00:53:50.669014 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e46c83-bf50-459b-b2cf-815672194549 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:50.732795 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:50.764227 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e46c83-bf50-459b-b2cf-815672194549 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:50.797613 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea20c6d-1d32-4da8-9c58-395d96f95ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:50.798224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea20c6d-1d32-4da8-9c58-395d96f95ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:50.798506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea20c6d-1d32-4da8-9c58-395d96f95ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:50.798777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea20c6d-1d32-4da8-9c58-395d96f95ab5 None admin] PATH_INFO: `/v3/projects/5d01567e03b24c8d85d2badbbf6affe9` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:50.799900 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea20c6d-1d32-4da8-9c58-395d96f95ab5 None admin] RBAC: Authorizing `identity:get_project(project_id=5d01567e03b24c8d85d2badbbf6affe9)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:50.801316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea20c6d-1d32-4da8-9c58-395d96f95ab5 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:50.802901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 585/1184] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:53:50 2022] GET /identity/v3/projects/5d01567e03b24c8d85d2badbbf6affe9 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:50.809059 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1075ca6a-2151-4c91-8489-8ac47beab26d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:50.905348 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1075ca6a-2151-4c91-8489-8ac47beab26d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:50.937660 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115da12-8187-4750-a816-d1cdbe1f410a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:50.938254 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115da12-8187-4750-a816-d1cdbe1f410a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:50.938534 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115da12-8187-4750-a816-d1cdbe1f410a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:50.938808 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115da12-8187-4750-a816-d1cdbe1f410a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:50.939519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115da12-8187-4750-a816-d1cdbe1f410a None admin] RBAC: Adding query filter params (role.id=ada268ea63354a0297070369dfe3c6cc, scope.project.id=5d01567e03b24c8d85d2badbbf6affe9, user.id=75522e09da8046e9b1be5687def496f7) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:53:50.939856 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115da12-8187-4750-a816-d1cdbe1f410a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:50.941270 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115da12-8187-4750-a816-d1cdbe1f410a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:50.946455 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 600/1185] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:53:50 2022] GET /identity/v3/role_assignments?user.id=75522e09da8046e9b1be5687def496f7&scope.project.id=5d01567e03b24c8d85d2badbbf6affe9&role.id=ada268ea63354a0297070369dfe3c6cc => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:53:53.046871 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41805f20-7737-4b96-955f-a94a8d655fac None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:53.047407 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41805f20-7737-4b96-955f-a94a8d655fac None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:53.047407 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41805f20-7737-4b96-955f-a94a8d655fac None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:53.048780 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 586/1186] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:53:53.053861 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffa0db-1b41-4d29-afa9-e069d71038c5 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:53.054283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffa0db-1b41-4d29-afa9-e069d71038c5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:53.054404 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffa0db-1b41-4d29-afa9-e069d71038c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:53.072920 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a3ffa0db-1b41-4d29-afa9-e069d71038c5 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:53:53.074325 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a3ffa0db-1b41-4d29-afa9-e069d71038c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:53:53.102415 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:53.140896 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 601/1187] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:53:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 00:53:53.410249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41805f20-7737-4b96-955f-a94a8d655fac None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:53.512025 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41805f20-7737-4b96-955f-a94a8d655fac None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:53.545068 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73811687-db33-4e9f-a1c7-875581f1318d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:53.545685 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73811687-db33-4e9f-a1c7-875581f1318d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:53.545961 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73811687-db33-4e9f-a1c7-875581f1318d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:53.546230 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73811687-db33-4e9f-a1c7-875581f1318d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:53.548756 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:53.556818 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73811687-db33-4e9f-a1c7-875581f1318d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:53:53.562140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-73811687-db33-4e9f-a1c7-875581f1318d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:53:53.562140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:53:53.562140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:53:53.562140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:53:53.562140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:53:53.562140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:53:53.562140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:53:53.562140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:53:53.562140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:53:53.562140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:53:53.562140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:53:53.562140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:53:53.562140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:53:53.562140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:53:53.562140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:53:53.562140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:53:53.563291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:53:53.563291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:53:53.563291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:53:53.563291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:53:53.563291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:53:53.563291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:53:53.563291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:53:53.563291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:53:53.563291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:53:53.563291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:53:53.563291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:53:53.563291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:53:53.563291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:53:53.563291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:53:53.563291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:53:53.563291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:53:53.564323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:53:53.564323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:53:53.564323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:53:53.564323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:53:53.564323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:53:53.564323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:53:53.564323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:53:53.564323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:53:53.564323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:53:53.564323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:53:53.564323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:53:53.564323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:53:53.564323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 587/1188] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:53:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:53:53.569277 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ffa0db-1b41-4d29-afa9-e069d71038c5 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:53.665415 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ffa0db-1b41-4d29-afa9-e069d71038c5 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:53.698739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d5801-4430-43f3-9db5-31adfdf9b9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:53.699499 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d5801-4430-43f3-9db5-31adfdf9b9e8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:53.699812 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d5801-4430-43f3-9db5-31adfdf9b9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:53.700089 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d5801-4430-43f3-9db5-31adfdf9b9e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:53.700751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d5801-4430-43f3-9db5-31adfdf9b9e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:53:53.701060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d5801-4430-43f3-9db5-31adfdf9b9e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:53.702425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d5801-4430-43f3-9db5-31adfdf9b9e8 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:53.710209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 602/1189] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:53:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:53.716451 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73811687-db33-4e9f-a1c7-875581f1318d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:53.815273 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73811687-db33-4e9f-a1c7-875581f1318d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:53.847768 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3c891-7986-459c-bdf5-694376bc6f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:53.848384 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3c891-7986-459c-bdf5-694376bc6f6e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:53.848667 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3c891-7986-459c-bdf5-694376bc6f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:53.848945 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3c891-7986-459c-bdf5-694376bc6f6e None admin] PATH_INFO: `/v3/users/75522e09da8046e9b1be5687def496f7` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:53.850057 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b3c891-7986-459c-bdf5-694376bc6f6e None admin] RBAC: Authorizing `identity:get_user(user_id=75522e09da8046e9b1be5687def496f7)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:53.851488 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b3c891-7986-459c-bdf5-694376bc6f6e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:53.853082 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 588/1190] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:53:53 2022] GET /identity/v3/users/75522e09da8046e9b1be5687def496f7 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:53.859314 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d5801-4430-43f3-9db5-31adfdf9b9e8 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:53.888371 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:53.958819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d5801-4430-43f3-9db5-31adfdf9b9e8 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:53.991071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8d619-75c5-4559-83aa-bfe5728657de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:53.991720 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8d619-75c5-4559-83aa-bfe5728657de None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:53.992002 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8d619-75c5-4559-83aa-bfe5728657de None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:53.992275 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8d619-75c5-4559-83aa-bfe5728657de None admin] PATH_INFO: `/v3/projects/5d01567e03b24c8d85d2badbbf6affe9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:53.993388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8d619-75c5-4559-83aa-bfe5728657de None admin] RBAC: Authorizing `identity:get_project(project_id=5d01567e03b24c8d85d2badbbf6affe9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:53.994819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8d619-75c5-4559-83aa-bfe5728657de None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:53.997005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 603/1191] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:53:53 2022] GET /identity/v3/projects/5d01567e03b24c8d85d2badbbf6affe9 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:54.002938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3c891-7986-459c-bdf5-694376bc6f6e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:54.065164 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:54.101139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3c891-7986-459c-bdf5-694376bc6f6e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:54.134673 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d76a2-353c-4a86-92f7-56322e2ffa33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:54.135319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d76a2-353c-4a86-92f7-56322e2ffa33 None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:54.135651 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d76a2-353c-4a86-92f7-56322e2ffa33 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:54.136010 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d76a2-353c-4a86-92f7-56322e2ffa33 None admin] PATH_INFO: `/v3/projects/5d01567e03b24c8d85d2badbbf6affe9/users/75522e09da8046e9b1be5687def496f7/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:54.137958 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782d76a2-353c-4a86-92f7-56322e2ffa33 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d01567e03b24c8d85d2badbbf6affe9, user_id=75522e09da8046e9b1be5687def496f7, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:54.139667 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782d76a2-353c-4a86-92f7-56322e2ffa33 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:54.152099 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 589/1192] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 00:53:54 2022] PUT /identity/v3/projects/5d01567e03b24c8d85d2badbbf6affe9/users/75522e09da8046e9b1be5687def496f7/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:53:56.287563 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab4063-edff-4920-8a6a-03739a8c3f28 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:56.288129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab4063-edff-4920-8a6a-03739a8c3f28 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:56.288129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab4063-edff-4920-8a6a-03739a8c3f28 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:56.289480 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 604/1193] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:53:56.295355 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10acc2-3732-420b-a6f4-ca48149478cf None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:56.295788 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10acc2-3732-420b-a6f4-ca48149478cf None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:56.295950 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10acc2-3732-420b-a6f4-ca48149478cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:56.314027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-ce10acc2-3732-420b-a6f4-ca48149478cf None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:53:56.315211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ce10acc2-3732-420b-a6f4-ca48149478cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:53:56.382407 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 590/1194] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:53:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:53:56.658061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab4063-edff-4920-8a6a-03739a8c3f28 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:56.751525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:56.758712 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab4063-edff-4920-8a6a-03739a8c3f28 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:56.792304 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a970bec-aae0-4680-8220-f68c662562d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:56.792918 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a970bec-aae0-4680-8220-f68c662562d6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:56.793196 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a970bec-aae0-4680-8220-f68c662562d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:56.793463 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a970bec-aae0-4680-8220-f68c662562d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:56.803162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a970bec-aae0-4680-8220-f68c662562d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:53:56.808457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-8a970bec-aae0-4680-8220-f68c662562d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:53:56.808457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:53:56.808457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:53:56.808457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:53:56.808457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:53:56.808457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:53:56.808457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:53:56.808457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:53:56.808457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:53:56.808457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:53:56.808457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:53:56.808457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:53:56.808457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:53:56.808457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:53:56.808457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:53:56.808457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:53:56.809597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:53:56.809597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:53:56.809597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:53:56.809597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:53:56.809597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:53:56.809597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:53:56.809597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:53:56.809597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:53:56.809597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:53:56.809597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:53:56.809597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:53:56.809597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:53:56.809597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:53:56.809597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:53:56.809597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:53:56.809597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:53:56.810857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:53:56.810857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:53:56.810857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:53:56.810857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:53:56.810857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:53:56.810857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:53:56.810857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:53:56.810857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:53:56.810857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:53:56.810857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:53:56.810857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:53:56.810857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:53:56.810857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 605/1195] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:53:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:53:56.815581 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10acc2-3732-420b-a6f4-ca48149478cf None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:56.868366 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:56.912805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10acc2-3732-420b-a6f4-ca48149478cf None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:56.946113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28eaa6-249d-43cd-a01d-ef6e6d615ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:56.946713 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28eaa6-249d-43cd-a01d-ef6e6d615ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:56.946995 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28eaa6-249d-43cd-a01d-ef6e6d615ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:56.947275 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28eaa6-249d-43cd-a01d-ef6e6d615ce0 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:56.948033 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28eaa6-249d-43cd-a01d-ef6e6d615ce0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:53:56.948613 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28eaa6-249d-43cd-a01d-ef6e6d615ce0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:56.949757 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28eaa6-249d-43cd-a01d-ef6e6d615ce0 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:56.957515 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 591/1196] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:53:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:56.963521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a970bec-aae0-4680-8220-f68c662562d6 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:57.062070 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a970bec-aae0-4680-8220-f68c662562d6 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:57.095262 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae223d-0290-4484-80a1-11422ab945cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:57.095906 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae223d-0290-4484-80a1-11422ab945cf None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:57.096189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae223d-0290-4484-80a1-11422ab945cf None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:57.096463 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae223d-0290-4484-80a1-11422ab945cf None admin] PATH_INFO: `/v3/users/75522e09da8046e9b1be5687def496f7` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:57.097585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae223d-0290-4484-80a1-11422ab945cf None admin] RBAC: Authorizing `identity:get_user(user_id=75522e09da8046e9b1be5687def496f7)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:57.099234 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae223d-0290-4484-80a1-11422ab945cf None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:57.101121 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 606/1197] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 00:53:56 2022] GET /identity/v3/users/75522e09da8046e9b1be5687def496f7 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:57.107424 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28eaa6-249d-43cd-a01d-ef6e6d615ce0 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:57.204808 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28eaa6-249d-43cd-a01d-ef6e6d615ce0 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:57.220636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:57.238248 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a7234-bf2a-4407-afe3-28519a3fc6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:57.238845 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a7234-bf2a-4407-afe3-28519a3fc6fb None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:57.239127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a7234-bf2a-4407-afe3-28519a3fc6fb None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:57.239404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a7234-bf2a-4407-afe3-28519a3fc6fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:57.304990 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060a7234-bf2a-4407-afe3-28519a3fc6fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:57.306431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060a7234-bf2a-4407-afe3-28519a3fc6fb None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:57.402488 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 592/1198] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:53:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 00:53:57.408460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae223d-0290-4484-80a1-11422ab945cf None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:57.433815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:57.506722 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae223d-0290-4484-80a1-11422ab945cf None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:57.540237 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c320e55-81fd-4212-af63-333ac7e54357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:57.540866 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c320e55-81fd-4212-af63-333ac7e54357 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:57.541148 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c320e55-81fd-4212-af63-333ac7e54357 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:57.541423 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c320e55-81fd-4212-af63-333ac7e54357 None admin] PATH_INFO: `/v3/projects/5d01567e03b24c8d85d2badbbf6affe9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:57.543056 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c320e55-81fd-4212-af63-333ac7e54357 None admin] RBAC: Authorizing `identity:get_project(project_id=5d01567e03b24c8d85d2badbbf6affe9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:57.544528 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c320e55-81fd-4212-af63-333ac7e54357 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:57.546151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 607/1199] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 00:53:57 2022] GET /identity/v3/projects/5d01567e03b24c8d85d2badbbf6affe9 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:53:57.552284 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a7234-bf2a-4407-afe3-28519a3fc6fb None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:53:57.569996 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:53:57.650435 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a7234-bf2a-4407-afe3-28519a3fc6fb None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:53:57.685004 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eecadf-50d1-4354-9497-650c32ccc90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:53:57.685597 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eecadf-50d1-4354-9497-650c32ccc90b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:53:57.685878 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eecadf-50d1-4354-9497-650c32ccc90b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:53:57.686153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eecadf-50d1-4354-9497-650c32ccc90b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:53:57.686867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eecadf-50d1-4354-9497-650c32ccc90b None admin] RBAC: Adding query filter params (role.id=ada268ea63354a0297070369dfe3c6cc, scope.project.id=5d01567e03b24c8d85d2badbbf6affe9, user.id=75522e09da8046e9b1be5687def496f7) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:53:57.687179 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eecadf-50d1-4354-9497-650c32ccc90b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:53:57.688643 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eecadf-50d1-4354-9497-650c32ccc90b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:53:57.694412 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 593/1200] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:53:57 2022] GET /identity/v3/role_assignments?user.id=75522e09da8046e9b1be5687def496f7&scope.project.id=5d01567e03b24c8d85d2badbbf6affe9&role.id=ada268ea63354a0297070369dfe3c6cc => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:54:07.536001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=54 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:54:07.691687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=60 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:54:53.507921 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a9feb-b793-456e-8da7-fb595dae64e2 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:54:53.508813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a9feb-b793-456e-8da7-fb595dae64e2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:54:53.508813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a9feb-b793-456e-8da7-fb595dae64e2 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:54:53.509890 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 608/1201] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:54:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:54:53.515969 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5be1e-9913-49fe-a247-d832c5e455d8 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:54:53.516559 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5be1e-9913-49fe-a247-d832c5e455d8 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:54:53.516559 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5be1e-9913-49fe-a247-d832c5e455d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:54:53.539603 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-2ab5be1e-9913-49fe-a247-d832c5e455d8 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:54:53.539603 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2ab5be1e-9913-49fe-a247-d832c5e455d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:54:53.630496 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 594/1202] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:54:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 00:54:53.926603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a9feb-b793-456e-8da7-fb595dae64e2 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:54:54.053229 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a9feb-b793-456e-8da7-fb595dae64e2 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:54:54.102417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20a972-2997-4cc5-849a-2a1f83972b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:54:54.103288 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20a972-2997-4cc5-849a-2a1f83972b3c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:54:54.103596 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20a972-2997-4cc5-849a-2a1f83972b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:54:54.104126 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20a972-2997-4cc5-849a-2a1f83972b3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:54:54.120608 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea20a972-2997-4cc5-849a-2a1f83972b3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:54:54.122488 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea20a972-2997-4cc5-849a-2a1f83972b3c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:54:54.140819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-ea20a972-2997-4cc5-849a-2a1f83972b3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:54:54.140819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:54:54.140819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:54:54.140819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:54:54.140819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:54:54.140819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:54:54.140819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:54:54.140819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:54:54.140819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:54:54.140819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:54:54.140819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:54:54.140819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:54:54.140819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:54:54.140819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:54:54.140819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:54:54.140819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:54:54.146985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:54:54.146985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:54:54.146985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:54:54.146985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:54:54.146985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:54:54.146985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:54:54.146985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:54:54.146985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:54:54.146985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:54:54.146985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:54:54.146985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:54:54.146985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:54:54.146985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:54:54.146985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:54:54.146985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:54:54.146985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:54:54.146985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:54:54.148929 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:54:54.148929 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:54:54.148929 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:54:54.148929 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 609/1203] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:54:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:54:54.150155 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab5be1e-9913-49fe-a247-d832c5e455d8 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:54:54.280983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab5be1e-9913-49fe-a247-d832c5e455d8 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:54:54.319728 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:54:54.328278 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2190d41d-be9f-4bfd-bb57-2141e6c8f734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:54:54.329040 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190d41d-be9f-4bfd-bb57-2141e6c8f734 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:54:54.329566 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190d41d-be9f-4bfd-bb57-2141e6c8f734 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:54:54.330160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190d41d-be9f-4bfd-bb57-2141e6c8f734 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:54:54.330996 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2190d41d-be9f-4bfd-bb57-2141e6c8f734 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:54:54.331335 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2190d41d-be9f-4bfd-bb57-2141e6c8f734 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:54:54.332866 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2190d41d-be9f-4bfd-bb57-2141e6c8f734 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:54:54.346457 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 595/1204] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:54:54.353009 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20a972-2997-4cc5-849a-2a1f83972b3c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:54:54.433019 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:54:54.506733 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20a972-2997-4cc5-849a-2a1f83972b3c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:54:54.550304 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5c1f5-e826-4f71-be52-24e9f3b25f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:54:54.550971 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5c1f5-e826-4f71-be52-24e9f3b25f1f None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:54:54.551206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5c1f5-e826-4f71-be52-24e9f3b25f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:54:54.551449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5c1f5-e826-4f71-be52-24e9f3b25f1f None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:54:54.552157 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd5c1f5-e826-4f71-be52-24e9f3b25f1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:54:54.553587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd5c1f5-e826-4f71-be52-24e9f3b25f1f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:54:54.585254 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 610/1205] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 00:54:54 2022] POST /identity/v3/users => generated 251 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:54:57.190661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367412b-7595-4e23-b4d3-0c4da45430bd None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:54:57.191218 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367412b-7595-4e23-b4d3-0c4da45430bd None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:54:57.191218 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367412b-7595-4e23-b4d3-0c4da45430bd None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:54:57.193710 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 596/1206] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:54:57.198914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584da31-09f6-409b-b97d-e562e35a5732 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:54:57.199524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584da31-09f6-409b-b97d-e562e35a5732 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:54:57.199524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584da31-09f6-409b-b97d-e562e35a5732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:54:57.219982 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-6584da31-09f6-409b-b97d-e562e35a5732 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:54:57.221274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6584da31-09f6-409b-b97d-e562e35a5732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:54:57.299130 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 611/1207] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:54:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:54:57.586999 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367412b-7595-4e23-b4d3-0c4da45430bd None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:54:57.708743 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367412b-7595-4e23-b4d3-0c4da45430bd None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:54:57.750000 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f4636-270d-47fb-ba15-31829274b4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:54:57.750625 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f4636-270d-47fb-ba15-31829274b4da None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:54:57.750911 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f4636-270d-47fb-ba15-31829274b4da None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:54:57.751195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f4636-270d-47fb-ba15-31829274b4da None admin] PATH_INFO: `/v3/roles/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:54:57.763422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f4636-270d-47fb-ba15-31829274b4da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:54:57.768731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-737f4636-270d-47fb-ba15-31829274b4da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:54:57.768731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:54:57.768731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:54:57.768731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:54:57.768731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:54:57.768731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:54:57.768731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:54:57.768731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:54:57.768731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:54:57.768731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:54:57.768731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:54:57.768731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:54:57.768731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:54:57.768731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:54:57.768731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:54:57.768731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:54:57.771614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:54:57.771614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:54:57.771614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:54:57.771614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:54:57.771614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:54:57.771614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:54:57.771614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:54:57.771614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:54:57.771614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:54:57.771614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:54:57.771614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:54:57.771614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:54:57.771614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:54:57.771614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:54:57.771614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:54:57.771614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:54:57.772499 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:54:57.772499 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:54:57.772499 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:54:57.772499 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:54:57.772499 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:54:57.772499 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:54:57.772499 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:54:57.772499 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:54:57.772499 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:54:57.772499 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:54:57.772499 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:54:57.772499 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:54:57.772499 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 597/1208] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 00:54:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:54:57.776308 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584da31-09f6-409b-b97d-e562e35a5732 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:54:57.827152 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:54:57.892220 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584da31-09f6-409b-b97d-e562e35a5732 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:54:57.929600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f2f1f-4d19-42f2-a42e-2d3faf5ca72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:54:57.930249 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f2f1f-4d19-42f2-a42e-2d3faf5ca72e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:54:57.930546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f2f1f-4d19-42f2-a42e-2d3faf5ca72e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:54:57.930838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f2f1f-4d19-42f2-a42e-2d3faf5ca72e None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:54:57.931615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f2f1f-4d19-42f2-a42e-2d3faf5ca72e None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:54:57.932845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f2f1f-4d19-42f2-a42e-2d3faf5ca72e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:54:57.933283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f2f1f-4d19-42f2-a42e-2d3faf5ca72e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:54:57.947589 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 612/1209] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 00:54:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:54:57.948549 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f4636-270d-47fb-ba15-31829274b4da None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:54:58.006214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:54:58.065886 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f4636-270d-47fb-ba15-31829274b4da None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:54:58.104609 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b26cf-6ffa-4be3-b84a-2208603b0492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:54:58.105338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b26cf-6ffa-4be3-b84a-2208603b0492 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:54:58.105614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b26cf-6ffa-4be3-b84a-2208603b0492 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:54:58.105891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b26cf-6ffa-4be3-b84a-2208603b0492 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:54:58.118447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b26cf-6ffa-4be3-b84a-2208603b0492 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:54:58.119946 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b26cf-6ffa-4be3-b84a-2208603b0492 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:54:58.135785 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-002b26cf-6ffa-4be3-b84a-2208603b0492 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:54:58.135785 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:54:58.135785 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:54:58.135785 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:54:58.135785 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:54:58.135785 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:54:58.135785 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:54:58.135785 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:54:58.135785 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:54:58.135785 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:54:58.135785 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:54:58.135785 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:54:58.135785 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:54:58.135785 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:54:58.135785 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:54:58.135785 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:54:58.136841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:54:58.136841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:54:58.136841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:54:58.136841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:54:58.136841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:54:58.136841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:54:58.136841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:54:58.136841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:54:58.136841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:54:58.136841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:54:58.136841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:54:58.136841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:54:58.136841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:54:58.136841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:54:58.136841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:54:58.136841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:54:58.136841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:54:58.137786 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:54:58.137786 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:54:58.137786 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:54:58.143107 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f2f1f-4d19-42f2-a42e-2d3faf5ca72e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:54:58.143580 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 598/1210] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:54:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:54:58.252891 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:54:58.257888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f2f1f-4d19-42f2-a42e-2d3faf5ca72e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:54:58.297004 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc41535-0a6c-4bd0-a843-44b28ee64dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:54:58.297630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc41535-0a6c-4bd0-a843-44b28ee64dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:54:58.297909 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc41535-0a6c-4bd0-a843-44b28ee64dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:54:58.298229 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc41535-0a6c-4bd0-a843-44b28ee64dd3 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:54:58.298881 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc41535-0a6c-4bd0-a843-44b28ee64dd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:54:58.299182 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc41535-0a6c-4bd0-a843-44b28ee64dd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:54:58.300661 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc41535-0a6c-4bd0-a843-44b28ee64dd3 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:54:58.315928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 613/1211] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:54:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:54:58.322232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b26cf-6ffa-4be3-b84a-2208603b0492 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:54:58.463832 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b26cf-6ffa-4be3-b84a-2208603b0492 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:54:58.465461 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:54:58.505372 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28808e-243d-40f7-9ea3-13e8ee06aed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:54:58.506010 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28808e-243d-40f7-9ea3-13e8ee06aed2 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:54:58.506290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28808e-243d-40f7-9ea3-13e8ee06aed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:54:58.506565 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28808e-243d-40f7-9ea3-13e8ee06aed2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:54:58.515641 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28808e-243d-40f7-9ea3-13e8ee06aed2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:54:58.517207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28808e-243d-40f7-9ea3-13e8ee06aed2 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:54:58.529596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-cb28808e-243d-40f7-9ea3-13e8ee06aed2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 00:54:58.529596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:54:58.529596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:54:58.529596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:54:58.529596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:54:58.529596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:54:58.529596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:54:58.529596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:54:58.529596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:54:58.529596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:54:58.529596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:54:58.529596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:54:58.529596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:54:58.529596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:54:58.529596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:54:58.529596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:54:58.530742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:54:58.530742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:54:58.530742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:54:58.530742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:54:58.530742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:54:58.530742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:54:58.530742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:54:58.530742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:54:58.530742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:54:58.530742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:54:58.530742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:54:58.530742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:54:58.530742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:54:58.530742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:54:58.530742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:54:58.530742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:54:58.530742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:54:58.530742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:54:58.532178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:54:58.532178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:54:58.532178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:54:58.532178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:54:58.532178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:54:58.532178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:54:58.532178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:54:58.532178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:54:58.532178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 00:54:58.532178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:54:58.532178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 599/1212] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:54:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:54:58.536525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc41535-0a6c-4bd0-a843-44b28ee64dd3 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:54:58.644614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc41535-0a6c-4bd0-a843-44b28ee64dd3 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:54:58.688484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a356f8-f97e-41c7-8953-8249f29a0c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:54:58.689136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a356f8-f97e-41c7-8953-8249f29a0c4d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:54:58.689421 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a356f8-f97e-41c7-8953-8249f29a0c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:54:58.689727 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a356f8-f97e-41c7-8953-8249f29a0c4d None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:54:58.690456 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a356f8-f97e-41c7-8953-8249f29a0c4d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:54:58.690765 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a356f8-f97e-41c7-8953-8249f29a0c4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:54:58.692327 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a356f8-f97e-41c7-8953-8249f29a0c4d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:54:58.711000 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 614/1213] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Sep 14 00:54:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:54:58.717263 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28808e-243d-40f7-9ea3-13e8ee06aed2 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:54:58.834901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28808e-243d-40f7-9ea3-13e8ee06aed2 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:54:58.876423 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762cf89-1ff3-45cd-b26f-d91b79a3ec40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:54:58.877056 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762cf89-1ff3-45cd-b26f-d91b79a3ec40 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:54:58.877363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762cf89-1ff3-45cd-b26f-d91b79a3ec40 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:54:58.877640 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762cf89-1ff3-45cd-b26f-d91b79a3ec40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:54:58.912441 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:54:58.957843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762cf89-1ff3-45cd-b26f-d91b79a3ec40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:54:58.958710 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762cf89-1ff3-45cd-b26f-d91b79a3ec40 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:54:59.077865 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 600/1214] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:54:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 00:54:59.083729 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a356f8-f97e-41c7-8953-8249f29a0c4d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:54:59.109567 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:54:59.212828 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a356f8-f97e-41c7-8953-8249f29a0c4d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:54:59.254370 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435cac8-c792-4cd3-81e7-a08417360801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:54:59.255034 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435cac8-c792-4cd3-81e7-a08417360801 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:54:59.255302 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435cac8-c792-4cd3-81e7-a08417360801 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:54:59.255583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435cac8-c792-4cd3-81e7-a08417360801 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:54:59.268407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8435cac8-c792-4cd3-81e7-a08417360801 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:54:59.269874 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8435cac8-c792-4cd3-81e7-a08417360801 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:54:59.285972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-8435cac8-c792-4cd3-81e7-a08417360801 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:54:59.285972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:54:59.285972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:54:59.285972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:54:59.285972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:54:59.285972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:54:59.285972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:54:59.285972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:54:59.285972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:54:59.285972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:54:59.285972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:54:59.285972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:54:59.285972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:54:59.285972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:54:59.285972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:54:59.285972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:54:59.286966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:54:59.286966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:54:59.286966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:54:59.286966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:54:59.286966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:54:59.286966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:54:59.286966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:54:59.286966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:54:59.286966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:54:59.286966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:54:59.286966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:54:59.286966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:54:59.286966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:54:59.286966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:54:59.286966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:54:59.286966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:54:59.286966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:54:59.287924 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:54:59.287924 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:54:59.287924 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:54:59.291930 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 615/1215] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:54:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:54:59.293661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762cf89-1ff3-45cd-b26f-d91b79a3ec40 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:54:59.367690 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:54:59.411309 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762cf89-1ff3-45cd-b26f-d91b79a3ec40 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:54:59.458220 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e97788-d607-4291-ad27-50071394470b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:54:59.458851 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e97788-d607-4291-ad27-50071394470b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:54:59.459137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e97788-d607-4291-ad27-50071394470b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:54:59.459428 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e97788-d607-4291-ad27-50071394470b None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:54:59.460484 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e97788-d607-4291-ad27-50071394470b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:54:59.460816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e97788-d607-4291-ad27-50071394470b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:54:59.462470 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e97788-d607-4291-ad27-50071394470b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:54:59.482473 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 601/1216] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:54:59.483465 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435cac8-c792-4cd3-81e7-a08417360801 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:54:59.567097 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:54:59.604906 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435cac8-c792-4cd3-81e7-a08417360801 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:54:59.642490 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f226057-1124-4b03-aea0-cf9cc7c1d165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:54:59.643056 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f226057-1124-4b03-aea0-cf9cc7c1d165 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:54:59.643335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f226057-1124-4b03-aea0-cf9cc7c1d165 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:54:59.643635 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f226057-1124-4b03-aea0-cf9cc7c1d165 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:54:59.651453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f226057-1124-4b03-aea0-cf9cc7c1d165 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:54:59.653040 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f226057-1124-4b03-aea0-cf9cc7c1d165 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:54:59.666838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-7f226057-1124-4b03-aea0-cf9cc7c1d165 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:54:59.666838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:54:59.666838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:54:59.666838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:54:59.666838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:54:59.666838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:54:59.666838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:54:59.666838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:54:59.666838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:54:59.666838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:54:59.666838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:54:59.666838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:54:59.666838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:54:59.666838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:54:59.666838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:54:59.667999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:54:59.667999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:54:59.667999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:54:59.667999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:54:59.667999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:54:59.667999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:54:59.667999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:54:59.667999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:54:59.667999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:54:59.667999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:54:59.667999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:54:59.667999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:54:59.667999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:54:59.667999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:54:59.667999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:54:59.667999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:54:59.667999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:54:59.669200 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:54:59.669200 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:54:59.669200 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:54:59.669200 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:54:59.669200 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:54:59.669200 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:54:59.669200 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:54:59.669200 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:54:59.669200 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 616/1217] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:54:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:54:59.675893 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e97788-d607-4291-ad27-50071394470b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:54:59.787812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e97788-d607-4291-ad27-50071394470b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:54:59.812492 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:54:59.825473 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9b030-0ba8-419b-bb0a-dc2fb84234d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:54:59.826084 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9b030-0ba8-419b-bb0a-dc2fb84234d8 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:54:59.826367 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9b030-0ba8-419b-bb0a-dc2fb84234d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:54:59.826644 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9b030-0ba8-419b-bb0a-dc2fb84234d8 None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:54:59.827336 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9b030-0ba8-419b-bb0a-dc2fb84234d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:54:59.827674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9b030-0ba8-419b-bb0a-dc2fb84234d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:54:59.829124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9b030-0ba8-419b-bb0a-dc2fb84234d8 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:54:59.841551 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 602/1218] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:54:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:54:59.847705 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f226057-1124-4b03-aea0-cf9cc7c1d165 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:54:59.962936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f226057-1124-4b03-aea0-cf9cc7c1d165 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:54:59.995987 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:00.002692 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ded95-3884-4550-801f-98a1d723e8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:00.003337 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ded95-3884-4550-801f-98a1d723e8fe None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:00.003717 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ded95-3884-4550-801f-98a1d723e8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:00.004026 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ded95-3884-4550-801f-98a1d723e8fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:00.004840 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ded95-3884-4550-801f-98a1d723e8fe None admin] RBAC: Adding query filter params (role.id=7a33c2ceb92a478f81753573f8ece1bd, scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3, user.id=18798838de284c84b5b838b3530a2c43) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:00.005202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ded95-3884-4550-801f-98a1d723e8fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:00.006674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ded95-3884-4550-801f-98a1d723e8fe None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:00.012899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 617/1219] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:54:59 2022] GET /identity/v3/role_assignments?user.id=18798838de284c84b5b838b3530a2c43&scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3&role.id=7a33c2ceb92a478f81753573f8ece1bd => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:02.298106 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a07be-f84e-4508-9c2f-4da7c94223af None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:02.298106 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a07be-f84e-4508-9c2f-4da7c94223af None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:02.298106 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a07be-f84e-4508-9c2f-4da7c94223af None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:02.298106 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 603/1220] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:55:02.304410 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994697a2-f72c-4353-a0f6-debff96ee3ec None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:02.304807 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994697a2-f72c-4353-a0f6-debff96ee3ec None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:02.305116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994697a2-f72c-4353-a0f6-debff96ee3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:02.326669 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-994697a2-f72c-4353-a0f6-debff96ee3ec None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:55:02.328135 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-994697a2-f72c-4353-a0f6-debff96ee3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:55:02.403195 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 618/1221] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:55:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:55:02.712199 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a07be-f84e-4508-9c2f-4da7c94223af None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:02.835252 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a07be-f84e-4508-9c2f-4da7c94223af None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:02.870455 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de6543-1672-447f-8f7f-742b8213ed17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:02.871097 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de6543-1672-447f-8f7f-742b8213ed17 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:02.871377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de6543-1672-447f-8f7f-742b8213ed17 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:02.871693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de6543-1672-447f-8f7f-742b8213ed17 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:02.882794 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de6543-1672-447f-8f7f-742b8213ed17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:55:02.887921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-38de6543-1672-447f-8f7f-742b8213ed17 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:55:02.887921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:02.887921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:02.887921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:02.887921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:02.887921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:02.887921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:02.887921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:02.887921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:02.887921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:02.887921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:02.887921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:02.887921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:55:02.887921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:55:02.887921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:55:02.887921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:55:02.891902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:55:02.891902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:55:02.891902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:55:02.891902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:55:02.891902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:02.891902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:02.891902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:02.891902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:02.891902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:02.891902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:02.891902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:02.891902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:02.891902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:02.891902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:02.891902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:02.891902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:02.893144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:02.893144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:02.893144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:02.893144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:02.893144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:55:02.893144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:55:02.893144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:55:02.893144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:55:02.893144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:55:02.893144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:55:02.893144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:55:02.893144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:55:02.893144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 604/1222] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 00:55:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:02.896870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994697a2-f72c-4353-a0f6-debff96ee3ec None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:03.012137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994697a2-f72c-4353-a0f6-debff96ee3ec None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:03.041622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:03.053957 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baecf78-8143-48cc-ad8d-6c2d04282678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:03.054596 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baecf78-8143-48cc-ad8d-6c2d04282678 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:03.054883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baecf78-8143-48cc-ad8d-6c2d04282678 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:03.055158 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baecf78-8143-48cc-ad8d-6c2d04282678 None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:03.055928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baecf78-8143-48cc-ad8d-6c2d04282678 None admin] RBAC: Adding query filter params (name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:03.056171 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baecf78-8143-48cc-ad8d-6c2d04282678 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:03.057542 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baecf78-8143-48cc-ad8d-6c2d04282678 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:03.067233 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 619/1223] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 00:55:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:03.074793 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de6543-1672-447f-8f7f-742b8213ed17 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:03.124907 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:03.192603 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de6543-1672-447f-8f7f-742b8213ed17 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:03.229311 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186e31b-c773-42f2-bba0-19d3aee1eb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:03.229950 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186e31b-c773-42f2-bba0-19d3aee1eb30 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:03.230231 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186e31b-c773-42f2-bba0-19d3aee1eb30 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:03.230504 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186e31b-c773-42f2-bba0-19d3aee1eb30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:03.240528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f186e31b-c773-42f2-bba0-19d3aee1eb30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:03.242679 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f186e31b-c773-42f2-bba0-19d3aee1eb30 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:03.257607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-f186e31b-c773-42f2-bba0-19d3aee1eb30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:55:03.257607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:03.257607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:03.257607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:03.257607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:03.257607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:03.257607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:03.257607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:03.257607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:03.257607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:03.257607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:03.257607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:03.257607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:55:03.257607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:55:03.257607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:55:03.257607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:55:03.258836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:03.258836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:03.258836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:03.258836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:03.258836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:03.258836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:03.258836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:03.258836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:03.258836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:03.258836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:03.258836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:03.258836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:03.258836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:03.258836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:03.258836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:03.258836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:03.258836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:55:03.259907 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:55:03.259907 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:55:03.259907 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:55:03.259907 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 605/1224] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:55:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:03.265015 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baecf78-8143-48cc-ad8d-6c2d04282678 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:03.382710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baecf78-8143-48cc-ad8d-6c2d04282678 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:03.419118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13996a0-533e-4890-9614-1a77b59446b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:03.419763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13996a0-533e-4890-9614-1a77b59446b4 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:03.420056 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13996a0-533e-4890-9614-1a77b59446b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:03.420334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13996a0-533e-4890-9614-1a77b59446b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:03.420976 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13996a0-533e-4890-9614-1a77b59446b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:03.421282 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13996a0-533e-4890-9614-1a77b59446b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:03.422669 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13996a0-533e-4890-9614-1a77b59446b4 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:03.434827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 620/1225] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:03.441841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186e31b-c773-42f2-bba0-19d3aee1eb30 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:03.548892 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:03.556385 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186e31b-c773-42f2-bba0-19d3aee1eb30 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:03.594783 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911874e5-e765-4642-9c4c-aecfb79837fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:03.595429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911874e5-e765-4642-9c4c-aecfb79837fc None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:03.595761 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911874e5-e765-4642-9c4c-aecfb79837fc None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:03.596068 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911874e5-e765-4642-9c4c-aecfb79837fc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:03.604079 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911874e5-e765-4642-9c4c-aecfb79837fc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:03.605543 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911874e5-e765-4642-9c4c-aecfb79837fc None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:03.618026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-911874e5-e765-4642-9c4c-aecfb79837fc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 00:55:03.618026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:03.618026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:03.618026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:03.618026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:03.618026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:03.618026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:03.618026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:03.618026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:03.618026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:03.618026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:03.618026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:03.618026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:55:03.618026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:55:03.618026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:55:03.618026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:55:03.619126 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:03.619126 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:03.619126 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:55:03.619126 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:55:03.619126 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:55:03.619126 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:55:03.619126 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:03.619126 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:03.619126 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:03.619126 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:03.619126 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:03.619126 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:03.619126 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:03.619126 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:03.619126 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:03.619126 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:03.619126 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:03.619126 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:03.620405 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:03.620405 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:03.620405 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:55:03.620405 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:55:03.620405 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:55:03.620405 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:55:03.620405 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:55:03.620405 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:55:03.620405 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 00:55:03.620405 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:55:03.623971 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 606/1226] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:55:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:03.625175 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13996a0-533e-4890-9614-1a77b59446b4 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:03.675428 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:03.735763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13996a0-533e-4890-9614-1a77b59446b4 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:03.772011 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921bcb6-3a1f-474d-bd65-7d216b45018a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:03.773402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921bcb6-3a1f-474d-bd65-7d216b45018a None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:03.773685 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921bcb6-3a1f-474d-bd65-7d216b45018a None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:03.773958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921bcb6-3a1f-474d-bd65-7d216b45018a None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:03.774688 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921bcb6-3a1f-474d-bd65-7d216b45018a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:03.774992 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921bcb6-3a1f-474d-bd65-7d216b45018a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:03.776542 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921bcb6-3a1f-474d-bd65-7d216b45018a None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:03.795247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 621/1227] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Sep 14 00:55:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:03.801648 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911874e5-e765-4642-9c4c-aecfb79837fc None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:03.919811 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911874e5-e765-4642-9c4c-aecfb79837fc None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:03.962448 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aacab12-3343-4cb6-a3dc-4f4760231ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:03.963115 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacab12-3343-4cb6-a3dc-4f4760231ac0 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:03.963394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacab12-3343-4cb6-a3dc-4f4760231ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:03.963701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacab12-3343-4cb6-a3dc-4f4760231ac0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:03.975761 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aacab12-3343-4cb6-a3dc-4f4760231ac0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:03.977579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aacab12-3343-4cb6-a3dc-4f4760231ac0 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:03.995398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-1aacab12-3343-4cb6-a3dc-4f4760231ac0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:55:03.995398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:03.995398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:03.995398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:03.995398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:03.995398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:03.995398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:03.995398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:03.995398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:03.995398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:03.995398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:03.995398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:03.995398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:55:03.995398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:55:03.995398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:55:03.995398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:55:03.996600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:03.996600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:03.996600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:03.996600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:03.996600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:03.996600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:03.996600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:03.996600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:03.996600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:03.996600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:03.996600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:03.996600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:03.996600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:03.996600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:03.996600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:03.996600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:03.996600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:55:03.997704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:55:03.997704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:55:03.997704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:55:03.997704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 607/1228] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:55:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:04.004554 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921bcb6-3a1f-474d-bd65-7d216b45018a None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:04.119061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921bcb6-3a1f-474d-bd65-7d216b45018a None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:04.121915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:04.164779 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b25b5-315e-4085-8e72-b03f3e2905b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:04.165532 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b25b5-315e-4085-8e72-b03f3e2905b9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:04.165870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b25b5-315e-4085-8e72-b03f3e2905b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:04.166325 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b25b5-315e-4085-8e72-b03f3e2905b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:04.167002 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b25b5-315e-4085-8e72-b03f3e2905b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:04.167378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b25b5-315e-4085-8e72-b03f3e2905b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:04.168859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b25b5-315e-4085-8e72-b03f3e2905b9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:04.181281 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 622/1229] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:55:04.187704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aacab12-3343-4cb6-a3dc-4f4760231ac0 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:04.199965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:04.300609 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aacab12-3343-4cb6-a3dc-4f4760231ac0 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:04.337521 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d8cd8-227c-4bd9-b531-6127cc48c757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:04.338166 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d8cd8-227c-4bd9-b531-6127cc48c757 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:04.338450 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d8cd8-227c-4bd9-b531-6127cc48c757 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:04.338725 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d8cd8-227c-4bd9-b531-6127cc48c757 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:04.346283 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d8cd8-227c-4bd9-b531-6127cc48c757 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:04.347874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d8cd8-227c-4bd9-b531-6127cc48c757 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:04.361200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-3b9d8cd8-227c-4bd9-b531-6127cc48c757 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:55:04.361200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:04.361200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:04.361200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:04.361200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:04.361200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:04.361200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:04.361200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:04.361200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:04.361200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:04.361200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:04.361200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:04.361200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:55:04.361200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:55:04.361200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:55:04.362398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:55:04.362398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:04.362398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:04.362398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:04.362398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:04.362398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:04.362398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:04.362398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:04.362398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:04.362398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:04.362398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:04.362398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:04.362398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:04.362398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:04.362398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:04.362398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:04.362398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:04.363664 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:55:04.363664 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:55:04.363664 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:55:04.363664 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:55:04.363664 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:55:04.363664 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:55:04.363664 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:55:04.363664 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:55:04.363664 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 608/1230] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:55:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:04.369038 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b25b5-315e-4085-8e72-b03f3e2905b9 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:04.478464 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b25b5-315e-4085-8e72-b03f3e2905b9 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:04.518365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165f17d-bf40-466e-bf20-ed0856f57561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:04.519093 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165f17d-bf40-466e-bf20-ed0856f57561 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:04.519316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165f17d-bf40-466e-bf20-ed0856f57561 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:04.519616 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165f17d-bf40-466e-bf20-ed0856f57561 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:04.520357 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165f17d-bf40-466e-bf20-ed0856f57561 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:04.520670 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165f17d-bf40-466e-bf20-ed0856f57561 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:04.522281 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165f17d-bf40-466e-bf20-ed0856f57561 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:04.537133 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 623/1231] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:55:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:04.543861 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d8cd8-227c-4bd9-b531-6127cc48c757 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:04.614929 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:04.656607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d8cd8-227c-4bd9-b531-6127cc48c757 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:04.692226 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c7dee-51c9-4da9-acf4-cd1037922c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:04.692882 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c7dee-51c9-4da9-acf4-cd1037922c5b None admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:04.693164 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c7dee-51c9-4da9-acf4-cd1037922c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:04.693440 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c7dee-51c9-4da9-acf4-cd1037922c5b None admin] PATH_INFO: `/v3/projects/fda69e8c2d5e466dbc1244b2f76296a3/users/18798838de284c84b5b838b3530a2c43/roles/7a33c2ceb92a478f81753573f8ece1bd` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:04.708741 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c7dee-51c9-4da9-acf4-cd1037922c5b None admin] RBAC: Authorizing `identity:create_grant(project_id=fda69e8c2d5e466dbc1244b2f76296a3, user_id=18798838de284c84b5b838b3530a2c43, role_id=7a33c2ceb92a478f81753573f8ece1bd)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:04.710388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c7dee-51c9-4da9-acf4-cd1037922c5b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:04.722627 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 609/1232] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 00:55:04 2022] PUT /identity/v3/projects/fda69e8c2d5e466dbc1244b2f76296a3/users/18798838de284c84b5b838b3530a2c43/roles/7a33c2ceb92a478f81753573f8ece1bd => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 00:55:07.015772 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ba407-977e-48dd-9d7a-2350d70daee1 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:07.016336 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ba407-977e-48dd-9d7a-2350d70daee1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:07.016336 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ba407-977e-48dd-9d7a-2350d70daee1 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:07.017638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 624/1233] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:55:07.023450 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63895568-8c57-4cce-ab68-573db09585a5 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:07.023934 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63895568-8c57-4cce-ab68-573db09585a5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:07.024034 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63895568-8c57-4cce-ab68-573db09585a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:07.044478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-63895568-8c57-4cce-ab68-573db09585a5 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:55:07.045837 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-63895568-8c57-4cce-ab68-573db09585a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:55:07.131620 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 610/1234] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:55:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:55:07.396059 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ba407-977e-48dd-9d7a-2350d70daee1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:07.422840 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:07.510744 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ba407-977e-48dd-9d7a-2350d70daee1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:07.558092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92392f-2b21-4da2-ba1d-15d9eabf8dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:07.558819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92392f-2b21-4da2-ba1d-15d9eabf8dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:07.559133 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92392f-2b21-4da2-ba1d-15d9eabf8dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:07.559472 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92392f-2b21-4da2-ba1d-15d9eabf8dc1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:07.570419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa92392f-2b21-4da2-ba1d-15d9eabf8dc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:55:07.575201 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-fa92392f-2b21-4da2-ba1d-15d9eabf8dc1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:55:07.575201 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:07.575201 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:07.575201 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:07.575201 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:07.575201 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:07.575201 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:07.575201 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:07.575201 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:07.575201 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:07.575201 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:07.575201 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:07.575201 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:55:07.575201 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:55:07.575201 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:55:07.575201 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:55:07.576310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:55:07.576310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:55:07.576310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:55:07.576310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:55:07.576310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:07.576310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:07.576310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:07.576310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:07.576310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:07.576310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:07.576310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:07.576310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:07.576310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:07.576310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:07.576310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:07.576310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:07.577184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:07.577184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:07.577184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:07.577184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:07.577184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:55:07.577184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:55:07.577184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:55:07.577184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:55:07.577184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:55:07.577184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:55:07.577184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 00:55:07.577184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:55:07.577184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 625/1235] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 00:55:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:07.582525 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63895568-8c57-4cce-ab68-573db09585a5 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:07.634123 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:07.712676 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63895568-8c57-4cce-ab68-573db09585a5 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:07.774076 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1cd258-c27f-4439-a03f-460606428ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:07.774076 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1cd258-c27f-4439-a03f-460606428ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:07.774341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1cd258-c27f-4439-a03f-460606428ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:07.774688 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1cd258-c27f-4439-a03f-460606428ba5 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:07.779527 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1cd258-c27f-4439-a03f-460606428ba5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:07.779527 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1cd258-c27f-4439-a03f-460606428ba5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:07.779527 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1cd258-c27f-4439-a03f-460606428ba5 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:07.790860 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 611/1236] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 00:55:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:07.797628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92392f-2b21-4da2-ba1d-15d9eabf8dc1 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:07.902104 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:07.920207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92392f-2b21-4da2-ba1d-15d9eabf8dc1 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:07.960194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39464c36-019b-4e04-ac67-96a635a9f21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:07.960756 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39464c36-019b-4e04-ac67-96a635a9f21e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:07.961038 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39464c36-019b-4e04-ac67-96a635a9f21e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:07.961315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39464c36-019b-4e04-ac67-96a635a9f21e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:07.984598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39464c36-019b-4e04-ac67-96a635a9f21e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:07.986232 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39464c36-019b-4e04-ac67-96a635a9f21e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:08.003545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-39464c36-019b-4e04-ac67-96a635a9f21e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:55:08.003545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:08.003545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:08.003545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:08.003545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:08.003545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:08.003545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:08.003545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:08.003545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:08.003545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:08.003545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:08.003545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:08.003545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:55:08.003545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:55:08.003545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:55:08.003545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:55:08.004636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:08.004636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:08.004636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:08.004636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:08.004636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:08.004636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:08.004636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:08.004636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:08.004636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:08.004636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:08.004636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:08.004636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:08.004636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:08.004636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:08.004636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:08.004636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:08.004636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:55:08.008767 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:55:08.008767 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:55:08.008767 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:55:08.008767 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 626/1237] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:55:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:08.012059 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1cd258-c27f-4439-a03f-460606428ba5 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:08.136886 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1cd258-c27f-4439-a03f-460606428ba5 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:08.148359 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:08.179152 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe255e67-bc75-4be4-8095-562568924acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:08.179782 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe255e67-bc75-4be4-8095-562568924acb None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:08.180067 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe255e67-bc75-4be4-8095-562568924acb None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:08.180343 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe255e67-bc75-4be4-8095-562568924acb None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:08.180990 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe255e67-bc75-4be4-8095-562568924acb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:08.181297 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe255e67-bc75-4be4-8095-562568924acb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:08.182683 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe255e67-bc75-4be4-8095-562568924acb None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:08.201131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 612/1238] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:55:08.202362 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39464c36-019b-4e04-ac67-96a635a9f21e None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:08.314187 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39464c36-019b-4e04-ac67-96a635a9f21e None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:08.355097 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc7030-f8ce-4d31-a714-516ec1255ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:08.355763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc7030-f8ce-4d31-a714-516ec1255ecf None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:08.356055 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc7030-f8ce-4d31-a714-516ec1255ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:08.356312 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc7030-f8ce-4d31-a714-516ec1255ecf None admin] PATH_INFO: `/v3/users/placement` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:08.359111 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:08.365341 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc7030-f8ce-4d31-a714-516ec1255ecf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:08.367139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc7030-f8ce-4d31-a714-516ec1255ecf None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:08.378353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-cddc7030-f8ce-4d31-a714-516ec1255ecf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 00:55:08.378353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:08.378353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:08.378353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:08.378353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:08.378353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:08.378353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:08.378353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:08.378353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:08.378353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:08.378353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:08.378353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:08.378353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:55:08.378353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:55:08.378353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:55:08.378353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:55:08.379492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:08.379492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:08.379492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:55:08.379492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:55:08.379492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:55:08.379492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:55:08.379492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:08.379492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:08.379492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:08.379492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:08.379492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:08.379492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:08.379492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:08.379492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:08.379492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:08.379492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:08.379492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:08.379492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:08.380512 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:08.380512 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:08.380512 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:55:08.380512 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:55:08.380512 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:55:08.380512 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:55:08.380512 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:55:08.380512 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:55:08.380512 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 00:55:08.380512 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:55:08.380512 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 627/1239] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:55:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:08.386289 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe255e67-bc75-4be4-8095-562568924acb None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:08.507738 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe255e67-bc75-4be4-8095-562568924acb None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:08.547216 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ac262-c83f-4400-a8b1-49bead301b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:08.547865 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ac262-c83f-4400-a8b1-49bead301b0d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:08.548151 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ac262-c83f-4400-a8b1-49bead301b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:08.548451 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ac262-c83f-4400-a8b1-49bead301b0d None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:08.549292 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ac262-c83f-4400-a8b1-49bead301b0d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:08.549603 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ac262-c83f-4400-a8b1-49bead301b0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:08.551268 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ac262-c83f-4400-a8b1-49bead301b0d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:08.577178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 613/1240] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Sep 14 00:55:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:08.578159 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc7030-f8ce-4d31-a714-516ec1255ecf None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:08.689462 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc7030-f8ce-4d31-a714-516ec1255ecf None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:08.729442 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92629804-31cd-4afd-843c-4d463c495f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:08.730185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92629804-31cd-4afd-843c-4d463c495f26 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:08.730632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92629804-31cd-4afd-843c-4d463c495f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:08.731114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92629804-31cd-4afd-843c-4d463c495f26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:08.806442 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:08.814116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92629804-31cd-4afd-843c-4d463c495f26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:08.815540 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92629804-31cd-4afd-843c-4d463c495f26 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:08.932204 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 628/1241] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:55:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 14 00:55:08.938669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ac262-c83f-4400-a8b1-49bead301b0d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:08.969098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:09.048900 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ac262-c83f-4400-a8b1-49bead301b0d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:09.088170 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216c7fd-6f78-4a57-939f-d5f4cce1013b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:09.089001 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216c7fd-6f78-4a57-939f-d5f4cce1013b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:09.089212 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216c7fd-6f78-4a57-939f-d5f4cce1013b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:09.089522 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216c7fd-6f78-4a57-939f-d5f4cce1013b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:09.103236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216c7fd-6f78-4a57-939f-d5f4cce1013b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:09.104843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216c7fd-6f78-4a57-939f-d5f4cce1013b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:09.124453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-5216c7fd-6f78-4a57-939f-d5f4cce1013b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:55:09.124453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:09.124453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:09.124453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:09.124453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:09.124453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:09.124453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:09.124453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:09.124453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:09.124453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:09.124453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:09.124453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:09.124453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:55:09.124453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:55:09.124453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:55:09.124453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:55:09.125777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:09.125777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:09.125777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:09.125777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:09.125777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:09.125777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:09.125777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:09.125777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:09.125777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:09.125777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:09.125777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:09.125777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:09.125777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:09.125777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:09.125777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:09.125777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:09.125777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:55:09.126937 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:55:09.126937 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:55:09.126937 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:55:09.126937 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 614/1242] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:55:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:09.132846 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92629804-31cd-4afd-843c-4d463c495f26 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:09.237921 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:09.244714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92629804-31cd-4afd-843c-4d463c495f26 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:09.282221 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85d880-2c33-4404-a57a-3e43fb894ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:09.282926 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85d880-2c33-4404-a57a-3e43fb894ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:09.283278 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85d880-2c33-4404-a57a-3e43fb894ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:09.283548 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85d880-2c33-4404-a57a-3e43fb894ad6 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:09.284214 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85d880-2c33-4404-a57a-3e43fb894ad6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:09.284520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85d880-2c33-4404-a57a-3e43fb894ad6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:09.286048 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85d880-2c33-4404-a57a-3e43fb894ad6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:09.298756 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 629/1243] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:09.304621 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216c7fd-6f78-4a57-939f-d5f4cce1013b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:09.391895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:09.418268 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216c7fd-6f78-4a57-939f-d5f4cce1013b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:09.456354 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1a49b-8ebd-493e-83fe-6f7ff7e5ab31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:09.457031 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1a49b-8ebd-493e-83fe-6f7ff7e5ab31 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:09.457353 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1a49b-8ebd-493e-83fe-6f7ff7e5ab31 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:09.457706 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1a49b-8ebd-493e-83fe-6f7ff7e5ab31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:09.465776 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da1a49b-8ebd-493e-83fe-6f7ff7e5ab31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:09.467258 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da1a49b-8ebd-493e-83fe-6f7ff7e5ab31 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:09.480376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-8da1a49b-8ebd-493e-83fe-6f7ff7e5ab31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:55:09.480376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:09.480376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:09.480376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:09.480376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:09.480376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:09.480376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:09.480376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:09.480376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:09.480376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:09.480376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:09.480376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:09.480376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:55:09.480376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:55:09.480376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:55:09.481606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:55:09.481606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:09.481606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:09.481606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:09.481606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:09.481606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:09.481606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:09.481606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:09.481606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:09.481606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:09.481606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:09.481606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:09.481606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:09.481606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:09.481606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:09.481606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:09.481606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:09.488751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:55:09.488751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:55:09.488751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:55:09.488751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:55:09.488751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:55:09.488751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:55:09.488751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:55:09.488751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:55:09.488751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 615/1244] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:55:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 00:55:09.489391 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85d880-2c33-4404-a57a-3e43fb894ad6 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:09.598153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85d880-2c33-4404-a57a-3e43fb894ad6 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:09.640946 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9cc7d1-5bdc-437e-8ed6-1557a986df14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:09.641600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cc7d1-5bdc-437e-8ed6-1557a986df14 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:09.641881 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cc7d1-5bdc-437e-8ed6-1557a986df14 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:09.642152 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cc7d1-5bdc-437e-8ed6-1557a986df14 None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:09.642849 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9cc7d1-5bdc-437e-8ed6-1557a986df14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:09.643161 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9cc7d1-5bdc-437e-8ed6-1557a986df14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:09.644651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9cc7d1-5bdc-437e-8ed6-1557a986df14 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:09.657037 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 630/1245] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:55:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:55:09.664794 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1a49b-8ebd-493e-83fe-6f7ff7e5ab31 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:09.783919 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1a49b-8ebd-493e-83fe-6f7ff7e5ab31 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:09.820201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0994920-c94a-4c2d-8ddb-e70d12c6cc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:09.820985 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0994920-c94a-4c2d-8ddb-e70d12c6cc6c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:09.821250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0994920-c94a-4c2d-8ddb-e70d12c6cc6c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:09.821532 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0994920-c94a-4c2d-8ddb-e70d12c6cc6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:09.822266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0994920-c94a-4c2d-8ddb-e70d12c6cc6c None admin] RBAC: Adding query filter params (role.id=7a33c2ceb92a478f81753573f8ece1bd, scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3, user.id=18798838de284c84b5b838b3530a2c43) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:09.822575 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0994920-c94a-4c2d-8ddb-e70d12c6cc6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:09.824054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0994920-c94a-4c2d-8ddb-e70d12c6cc6c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:09.830069 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 616/1246] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:55:09 2022] GET /identity/v3/role_assignments?user.id=18798838de284c84b5b838b3530a2c43&scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3&role.id=7a33c2ceb92a478f81753573f8ece1bd => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:55:12.199303 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439dd8ac-1452-4160-b452-aa30296dd4d1 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:12.199940 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439dd8ac-1452-4160-b452-aa30296dd4d1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:12.200042 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439dd8ac-1452-4160-b452-aa30296dd4d1 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:12.201501 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 631/1247] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:55:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:55:12.206931 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62546581-c555-477f-8f5f-5736d5925e87 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:12.207368 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62546581-c555-477f-8f5f-5736d5925e87 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:12.207495 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62546581-c555-477f-8f5f-5736d5925e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:12.214196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:12.228792 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-62546581-c555-477f-8f5f-5736d5925e87 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:55:12.230401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-62546581-c555-477f-8f5f-5736d5925e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:55:12.307885 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 617/1248] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:55:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:55:12.589888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439dd8ac-1452-4160-b452-aa30296dd4d1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:12.609890 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:12.700752 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439dd8ac-1452-4160-b452-aa30296dd4d1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:12.736919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a766ac-e53d-4592-90dc-0154ec1d40ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:12.737485 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a766ac-e53d-4592-90dc-0154ec1d40ca None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:12.737766 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a766ac-e53d-4592-90dc-0154ec1d40ca None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:12.738040 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a766ac-e53d-4592-90dc-0154ec1d40ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:12.750474 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a766ac-e53d-4592-90dc-0154ec1d40ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:55:12.755408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-26a766ac-e53d-4592-90dc-0154ec1d40ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:55:12.755408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:12.755408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:12.755408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:12.755408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:12.755408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:12.755408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:12.755408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:12.755408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:12.755408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:12.755408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:12.755408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:12.755408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:55:12.755408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:55:12.755408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:55:12.755408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:55:12.756508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:55:12.756508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:55:12.756508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:55:12.756508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:55:12.756508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:12.756508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:12.756508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:12.756508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:12.756508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:12.756508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:12.756508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:12.756508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:12.756508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:12.756508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:12.756508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:12.756508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:12.757388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:12.757388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:12.757388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:12.757388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:12.757388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:55:12.757388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:55:12.757388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:55:12.757388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:55:12.757388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:55:12.757388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:55:12.757388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:55:12.757388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:55:12.757388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 632/1249] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:55:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:12.763173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62546581-c555-477f-8f5f-5736d5925e87 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:12.868879 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62546581-c555-477f-8f5f-5736d5925e87 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:12.900666 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:12.906170 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fc6b4-64b0-49d6-a5df-d62603a869b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:12.906795 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fc6b4-64b0-49d6-a5df-d62603a869b6 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:12.907072 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fc6b4-64b0-49d6-a5df-d62603a869b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:12.907343 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fc6b4-64b0-49d6-a5df-d62603a869b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:12.908047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410fc6b4-64b0-49d6-a5df-d62603a869b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:12.908349 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410fc6b4-64b0-49d6-a5df-d62603a869b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:12.909737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410fc6b4-64b0-49d6-a5df-d62603a869b6 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:12.919663 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 618/1250] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:55:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:12.925808 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a766ac-e53d-4592-90dc-0154ec1d40ca None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:13.005735 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:13.030827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a766ac-e53d-4592-90dc-0154ec1d40ca None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:13.066804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b8301-6020-40ae-bb52-13f25a0b2b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:13.067439 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b8301-6020-40ae-bb52-13f25a0b2b7d None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:13.067785 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b8301-6020-40ae-bb52-13f25a0b2b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:13.068064 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b8301-6020-40ae-bb52-13f25a0b2b7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:13.078088 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b8301-6020-40ae-bb52-13f25a0b2b7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:13.079538 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b8301-6020-40ae-bb52-13f25a0b2b7d None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:13.094470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-7b3b8301-6020-40ae-bb52-13f25a0b2b7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:55:13.094470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:13.094470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:13.094470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:13.094470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:13.094470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:13.094470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:13.094470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:13.094470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:13.094470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:13.094470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:13.094470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:13.094470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:55:13.094470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:55:13.094470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:55:13.094470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:55:13.095723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:13.095723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:13.095723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:13.095723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:13.095723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:13.095723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:13.095723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:13.095723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:13.095723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:13.095723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:13.095723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:13.095723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:13.095723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:13.095723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:13.095723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:13.095723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:13.095723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:55:13.096937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:55:13.096937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:55:13.096937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:55:13.096937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 633/1251] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:55:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:13.102435 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fc6b4-64b0-49d6-a5df-d62603a869b6 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:13.219766 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fc6b4-64b0-49d6-a5df-d62603a869b6 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:13.253152 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c0975-4ea0-43f7-b335-abb4925f47bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:13.253755 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c0975-4ea0-43f7-b335-abb4925f47bc None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:13.254035 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c0975-4ea0-43f7-b335-abb4925f47bc None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:13.254311 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c0975-4ea0-43f7-b335-abb4925f47bc None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:13.254948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c0975-4ea0-43f7-b335-abb4925f47bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:13.255252 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c0975-4ea0-43f7-b335-abb4925f47bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:13.257347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c0975-4ea0-43f7-b335-abb4925f47bc None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:13.269149 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 619/1252] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:55:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:13.274901 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b8301-6020-40ae-bb52-13f25a0b2b7d None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:13.373597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b8301-6020-40ae-bb52-13f25a0b2b7d None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:13.398993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:13.406781 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6890e1-bafd-43e2-8c8c-808729ca9fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:13.407399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6890e1-bafd-43e2-8c8c-808729ca9fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:13.407720 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6890e1-bafd-43e2-8c8c-808729ca9fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:13.407994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6890e1-bafd-43e2-8c8c-808729ca9fd0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:13.415739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6890e1-bafd-43e2-8c8c-808729ca9fd0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:13.417163 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6890e1-bafd-43e2-8c8c-808729ca9fd0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:13.428571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-ee6890e1-bafd-43e2-8c8c-808729ca9fd0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 00:55:13.428571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:13.428571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:13.428571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:13.428571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:13.428571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:13.428571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:13.428571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:13.428571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:13.428571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:13.428571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:13.428571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:13.428571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:55:13.428571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:55:13.428571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:55:13.428571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:55:13.429711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:13.429711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:13.429711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:55:13.429711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:55:13.429711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:55:13.429711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:55:13.429711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:13.429711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:13.429711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:13.429711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:13.429711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:13.429711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:13.429711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:13.429711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:13.429711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:13.429711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:13.429711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:13.429711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:13.430993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:13.430993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:13.430993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:55:13.430993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:55:13.430993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:55:13.430993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:55:13.430993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:55:13.430993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:55:13.430993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 00:55:13.430993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:55:13.430993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 634/1253] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:55:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:13.435608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c0975-4ea0-43f7-b335-abb4925f47bc None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:13.489716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:13.533955 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c0975-4ea0-43f7-b335-abb4925f47bc None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:13.567775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14033190-9d41-490b-8b45-d95304a159dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:13.568377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14033190-9d41-490b-8b45-d95304a159dd None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:13.568701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14033190-9d41-490b-8b45-d95304a159dd None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:13.568977 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14033190-9d41-490b-8b45-d95304a159dd None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:13.569690 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14033190-9d41-490b-8b45-d95304a159dd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:13.569997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14033190-9d41-490b-8b45-d95304a159dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:13.571433 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14033190-9d41-490b-8b45-d95304a159dd None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:13.588125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 620/1254] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Sep 14 00:55:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:13.594024 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6890e1-bafd-43e2-8c8c-808729ca9fd0 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:13.693252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6890e1-bafd-43e2-8c8c-808729ca9fd0 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:13.726658 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717385a-74a8-4fff-8f37-d734bcfd7ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:13.727255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717385a-74a8-4fff-8f37-d734bcfd7ffb None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:13.727539 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717385a-74a8-4fff-8f37-d734bcfd7ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:13.727845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717385a-74a8-4fff-8f37-d734bcfd7ffb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:13.781110 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:13.794588 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717385a-74a8-4fff-8f37-d734bcfd7ffb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:13.796016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717385a-74a8-4fff-8f37-d734bcfd7ffb None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:13.891446 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 635/1255] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:55:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 00:55:13.898010 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14033190-9d41-490b-8b45-d95304a159dd None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:13.995151 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14033190-9d41-490b-8b45-d95304a159dd None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:14.008015 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:14.029453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed159a-5f65-4203-aa95-6855e27d3000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:14.030071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed159a-5f65-4203-aa95-6855e27d3000 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:14.030347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed159a-5f65-4203-aa95-6855e27d3000 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:14.030615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed159a-5f65-4203-aa95-6855e27d3000 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:14.041162 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed159a-5f65-4203-aa95-6855e27d3000 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:14.042601 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed159a-5f65-4203-aa95-6855e27d3000 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:14.057816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-66ed159a-5f65-4203-aa95-6855e27d3000 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:55:14.057816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:14.057816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:14.057816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:14.057816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:14.057816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:14.057816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:14.057816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:14.057816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:14.057816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:14.057816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:14.057816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:14.057816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:55:14.057816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:55:14.057816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:55:14.057816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:55:14.058820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:14.058820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:14.058820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:14.058820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:14.058820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:14.058820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:14.058820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:14.058820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:14.058820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:14.058820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:14.058820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:14.058820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:14.058820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:14.058820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:14.058820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:14.058820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:14.058820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:55:14.059758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:55:14.059758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:55:14.059758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:55:14.059758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 621/1256] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:55:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:14.064533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717385a-74a8-4fff-8f37-d734bcfd7ffb None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:14.156984 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:14.169822 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717385a-74a8-4fff-8f37-d734bcfd7ffb None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:14.202978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57485793-80c1-4798-8a04-cbda9429cd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:14.203599 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57485793-80c1-4798-8a04-cbda9429cd83 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:14.203880 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57485793-80c1-4798-8a04-cbda9429cd83 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:14.204150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57485793-80c1-4798-8a04-cbda9429cd83 None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:14.204775 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57485793-80c1-4798-8a04-cbda9429cd83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:14.205070 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57485793-80c1-4798-8a04-cbda9429cd83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:14.206422 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57485793-80c1-4798-8a04-cbda9429cd83 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:14.217956 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 636/1257] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:14.223546 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed159a-5f65-4203-aa95-6855e27d3000 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:14.321742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed159a-5f65-4203-aa95-6855e27d3000 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:14.354947 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45ded6-659b-48c0-b273-d28d3c5d4287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:14.355586 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45ded6-659b-48c0-b273-d28d3c5d4287 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:14.355861 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45ded6-659b-48c0-b273-d28d3c5d4287 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:14.356131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45ded6-659b-48c0-b273-d28d3c5d4287 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:14.362495 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45ded6-659b-48c0-b273-d28d3c5d4287 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:14.363954 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45ded6-659b-48c0-b273-d28d3c5d4287 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:14.374962 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-2d45ded6-659b-48c0-b273-d28d3c5d4287 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:55:14.374962 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:14.374962 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:14.374962 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:14.374962 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:14.374962 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:14.374962 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:14.374962 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:14.374962 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:14.374962 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:14.374962 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:14.374962 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:14.374962 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:55:14.374962 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:55:14.374962 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:55:14.376054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:55:14.376054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:14.376054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:14.376054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:14.376054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:14.376054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:14.376054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:14.376054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:14.376054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:14.376054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:14.376054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:14.376054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:14.376054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:14.376054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:14.376054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:14.376054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:14.376054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:14.377004 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:55:14.377004 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:55:14.377004 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:55:14.377004 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:55:14.377004 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:55:14.377004 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:55:14.377004 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:55:14.377004 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:55:14.377004 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 622/1258] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:55:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:14.382379 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57485793-80c1-4798-8a04-cbda9429cd83 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:14.479181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57485793-80c1-4798-8a04-cbda9429cd83 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:14.512689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f1ffe-554e-4ac7-ad75-df3125c67f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:14.513532 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f1ffe-554e-4ac7-ad75-df3125c67f0e None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:14.513811 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f1ffe-554e-4ac7-ad75-df3125c67f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:14.514082 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f1ffe-554e-4ac7-ad75-df3125c67f0e None admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:14.514755 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f1ffe-554e-4ac7-ad75-df3125c67f0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:14.515056 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f1ffe-554e-4ac7-ad75-df3125c67f0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:14.516492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f1ffe-554e-4ac7-ad75-df3125c67f0e None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:14.527886 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 637/1259] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:55:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:14.533871 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45ded6-659b-48c0-b273-d28d3c5d4287 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:14.559664 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:14.633079 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45ded6-659b-48c0-b273-d28d3c5d4287 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:14.666338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd17f98-77ad-405e-86bc-fe95174f578d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:14.666932 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd17f98-77ad-405e-86bc-fe95174f578d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:14.667213 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd17f98-77ad-405e-86bc-fe95174f578d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:14.667487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd17f98-77ad-405e-86bc-fe95174f578d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:14.668227 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd17f98-77ad-405e-86bc-fe95174f578d None admin] RBAC: Adding query filter params (role.id=ada268ea63354a0297070369dfe3c6cc, scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3, user.id=18798838de284c84b5b838b3530a2c43) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:14.668541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd17f98-77ad-405e-86bc-fe95174f578d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:14.669960 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd17f98-77ad-405e-86bc-fe95174f578d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:14.675103 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 623/1260] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:55:14 2022] GET /identity/v3/role_assignments?user.id=18798838de284c84b5b838b3530a2c43&scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3&role.id=ada268ea63354a0297070369dfe3c6cc => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:16.797617 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f02fe-19a9-4322-8ef7-37a2022764b6 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:16.798117 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f02fe-19a9-4322-8ef7-37a2022764b6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:16.798117 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f02fe-19a9-4322-8ef7-37a2022764b6 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:16.799440 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 638/1261] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:55:16.804595 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942f8a3-e015-4495-a07d-b4f6213b9fc4 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:16.804992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942f8a3-e015-4495-a07d-b4f6213b9fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:16.805144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942f8a3-e015-4495-a07d-b4f6213b9fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:16.823128 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a942f8a3-e015-4495-a07d-b4f6213b9fc4 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:55:16.824332 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a942f8a3-e015-4495-a07d-b4f6213b9fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:55:16.895289 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 624/1262] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:55:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:55:17.168118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f02fe-19a9-4322-8ef7-37a2022764b6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:17.179909 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:17.269616 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f02fe-19a9-4322-8ef7-37a2022764b6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:17.302404 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd63623-01e6-4ab7-98ab-420640de2693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:17.303714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd63623-01e6-4ab7-98ab-420640de2693 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:17.303998 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd63623-01e6-4ab7-98ab-420640de2693 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:17.304274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd63623-01e6-4ab7-98ab-420640de2693 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:17.313912 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd63623-01e6-4ab7-98ab-420640de2693 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:55:17.319659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-4fd63623-01e6-4ab7-98ab-420640de2693 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:55:17.319659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:17.319659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:17.319659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:17.319659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:17.319659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:17.319659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:17.319659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:17.319659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:17.319659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:17.319659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:17.319659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:17.319659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:55:17.319659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:55:17.319659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:55:17.319659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:55:17.320634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:55:17.320634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:55:17.320634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:55:17.320634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:55:17.320634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:17.320634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:17.320634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:17.320634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:17.320634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:17.320634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:17.320634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:17.320634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:17.320634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:17.320634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:17.320634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:17.320634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:17.321837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:17.321837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:17.321837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:17.321837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:17.321837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:55:17.321837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:55:17.321837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:55:17.321837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:55:17.321837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:55:17.321837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:55:17.321837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:55:17.321837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:55:17.321837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 639/1263] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:55:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:17.326727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942f8a3-e015-4495-a07d-b4f6213b9fc4 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:17.345137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:17.426981 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942f8a3-e015-4495-a07d-b4f6213b9fc4 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:17.460123 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5605ef-be9c-4593-9fe4-fbea15dcd2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:17.460714 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5605ef-be9c-4593-9fe4-fbea15dcd2f9 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:17.460994 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5605ef-be9c-4593-9fe4-fbea15dcd2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:17.461264 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5605ef-be9c-4593-9fe4-fbea15dcd2f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:17.461913 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5605ef-be9c-4593-9fe4-fbea15dcd2f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:17.462214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5605ef-be9c-4593-9fe4-fbea15dcd2f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:17.463607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5605ef-be9c-4593-9fe4-fbea15dcd2f9 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:17.471386 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 625/1264] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:55:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:17.477552 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd63623-01e6-4ab7-98ab-420640de2693 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:17.543531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:17.576668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd63623-01e6-4ab7-98ab-420640de2693 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:17.608689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85c455-8dde-45fc-981f-3a8812a6bdeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:17.609346 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85c455-8dde-45fc-981f-3a8812a6bdeb None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:17.609574 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85c455-8dde-45fc-981f-3a8812a6bdeb None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:17.609844 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85c455-8dde-45fc-981f-3a8812a6bdeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:17.620195 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85c455-8dde-45fc-981f-3a8812a6bdeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:17.621627 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85c455-8dde-45fc-981f-3a8812a6bdeb None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:17.635813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-4e85c455-8dde-45fc-981f-3a8812a6bdeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:55:17.635813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:17.635813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:17.635813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:17.635813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:17.635813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:17.635813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:17.635813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:17.635813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:17.635813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:17.635813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:17.635813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:17.635813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:55:17.635813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:55:17.635813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:55:17.635813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:55:17.641815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:17.641815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:17.641815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:17.641815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:17.641815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:17.641815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:17.641815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:17.641815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:17.641815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:17.641815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:17.641815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:17.641815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:17.641815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:17.641815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:17.641815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:17.641815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:17.641815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:55:17.642869 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:55:17.642869 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:55:17.642869 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:55:17.642869 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 640/1265] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:55:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:17.643159 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5605ef-be9c-4593-9fe4-fbea15dcd2f9 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:17.722906 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:17.742718 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5605ef-be9c-4593-9fe4-fbea15dcd2f9 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:17.775887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a1307-c19a-4a54-b58d-bfcf44f0517e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:17.776487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a1307-c19a-4a54-b58d-bfcf44f0517e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:17.776768 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a1307-c19a-4a54-b58d-bfcf44f0517e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:17.777042 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a1307-c19a-4a54-b58d-bfcf44f0517e None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:17.777671 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a1307-c19a-4a54-b58d-bfcf44f0517e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:17.777973 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a1307-c19a-4a54-b58d-bfcf44f0517e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:17.779344 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a1307-c19a-4a54-b58d-bfcf44f0517e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:17.792001 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 626/1266] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:55:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 00:55:17.797627 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85c455-8dde-45fc-981f-3a8812a6bdeb None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:17.894669 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85c455-8dde-45fc-981f-3a8812a6bdeb None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:17.913971 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:17.927887 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a3f81-0e86-45f1-a9f5-d7107ef6b49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:17.928637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a3f81-0e86-45f1-a9f5-d7107ef6b49c None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:17.928922 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a3f81-0e86-45f1-a9f5-d7107ef6b49c None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:17.929195 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a3f81-0e86-45f1-a9f5-d7107ef6b49c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:17.936354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a3f81-0e86-45f1-a9f5-d7107ef6b49c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:17.937783 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a3f81-0e86-45f1-a9f5-d7107ef6b49c None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:17.949061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-aa8a3f81-0e86-45f1-a9f5-d7107ef6b49c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 00:55:17.949061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:17.949061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:17.949061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:17.949061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:17.949061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:17.949061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:17.949061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:17.949061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:17.949061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:17.949061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:17.949061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:17.949061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:55:17.949061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:55:17.949061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:55:17.949061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:55:17.950139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:17.950139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:17.950139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:55:17.950139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:55:17.950139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:55:17.950139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:55:17.950139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:17.950139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:17.950139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:17.950139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:17.950139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:17.950139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:17.950139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:17.950139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:17.950139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:17.950139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:17.950139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:17.950139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:17.951458 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:17.951458 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:17.951458 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:55:17.951458 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:55:17.951458 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:55:17.951458 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:55:17.951458 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:55:17.951458 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:55:17.951458 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 00:55:17.951458 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:55:17.951458 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 641/1267] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:55:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:17.955869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a1307-c19a-4a54-b58d-bfcf44f0517e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:18.053701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a1307-c19a-4a54-b58d-bfcf44f0517e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:18.086322 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe6bc0-244a-496a-8fdc-dc91c28cb520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:18.086915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe6bc0-244a-496a-8fdc-dc91c28cb520 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:18.087195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe6bc0-244a-496a-8fdc-dc91c28cb520 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:18.087468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe6bc0-244a-496a-8fdc-dc91c28cb520 None admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:18.088213 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbe6bc0-244a-496a-8fdc-dc91c28cb520 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:18.088517 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbe6bc0-244a-496a-8fdc-dc91c28cb520 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:18.089937 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbe6bc0-244a-496a-8fdc-dc91c28cb520 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:18.098376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:18.109217 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 627/1268] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Sep 14 00:55:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:18.116341 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a3f81-0e86-45f1-a9f5-d7107ef6b49c None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:18.217121 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a3f81-0e86-45f1-a9f5-d7107ef6b49c None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:18.249317 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225aac6-b2c1-4c39-ae56-3e618f14ca15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:18.249931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225aac6-b2c1-4c39-ae56-3e618f14ca15 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:18.250207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225aac6-b2c1-4c39-ae56-3e618f14ca15 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:18.250477 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225aac6-b2c1-4c39-ae56-3e618f14ca15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:18.260497 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225aac6-b2c1-4c39-ae56-3e618f14ca15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:18.261941 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225aac6-b2c1-4c39-ae56-3e618f14ca15 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:18.276212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-c225aac6-b2c1-4c39-ae56-3e618f14ca15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:55:18.276212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:18.276212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:18.276212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:18.276212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:18.276212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:18.276212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:18.276212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:18.276212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:18.276212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:18.276212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:18.276212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:18.276212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:55:18.276212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:55:18.276212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:55:18.276212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:55:18.277189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:18.277189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:18.277189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:18.277189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:18.277189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:18.277189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:18.277189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:18.277189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:18.277189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:18.277189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:18.277189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:18.277189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:18.277189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:18.277189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:18.277189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:18.277189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:18.277189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:55:18.278225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:55:18.278225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:55:18.278225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:55:18.278225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 642/1269] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:55:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:18.283116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe6bc0-244a-496a-8fdc-dc91c28cb520 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:18.380545 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe6bc0-244a-496a-8fdc-dc91c28cb520 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:18.414827 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d087c99-43f7-42b1-877e-3f49e9f89371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:18.415425 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d087c99-43f7-42b1-877e-3f49e9f89371 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:18.415729 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d087c99-43f7-42b1-877e-3f49e9f89371 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:18.416002 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d087c99-43f7-42b1-877e-3f49e9f89371 None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:18.416630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d087c99-43f7-42b1-877e-3f49e9f89371 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:18.416934 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d087c99-43f7-42b1-877e-3f49e9f89371 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:18.418295 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d087c99-43f7-42b1-877e-3f49e9f89371 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:18.429804 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 628/1270] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:18.435407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225aac6-b2c1-4c39-ae56-3e618f14ca15 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:18.467692 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:18.535099 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225aac6-b2c1-4c39-ae56-3e618f14ca15 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:18.568054 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c864abd-2ee0-4a92-9ed4-7e3888329648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:18.568705 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c864abd-2ee0-4a92-9ed4-7e3888329648 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:18.568984 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c864abd-2ee0-4a92-9ed4-7e3888329648 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:18.569258 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c864abd-2ee0-4a92-9ed4-7e3888329648 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:18.575637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c864abd-2ee0-4a92-9ed4-7e3888329648 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:18.577070 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c864abd-2ee0-4a92-9ed4-7e3888329648 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:18.587845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-2c864abd-2ee0-4a92-9ed4-7e3888329648 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:55:18.587845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:18.587845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:18.587845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:18.587845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:18.587845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:18.587845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:18.587845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:18.587845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:18.587845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:18.587845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:18.587845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:18.587845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:55:18.587845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:55:18.587845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:55:18.588883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:55:18.588883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:18.588883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:18.588883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:18.588883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:18.588883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:18.588883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:18.588883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:18.588883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:18.588883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:18.588883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:18.588883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:18.588883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:18.588883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:18.588883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:18.588883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:18.588883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:18.590004 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:55:18.590004 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:55:18.590004 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:55:18.590004 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:55:18.590004 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:55:18.590004 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:55:18.590004 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:55:18.590004 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:55:18.590004 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 643/1271] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:55:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:18.594842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d087c99-43f7-42b1-877e-3f49e9f89371 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:18.652709 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:18.694505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d087c99-43f7-42b1-877e-3f49e9f89371 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:18.727413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8abca-66f7-4822-bd60-d49460386a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:18.728038 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8abca-66f7-4822-bd60-d49460386a9c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:18.728315 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8abca-66f7-4822-bd60-d49460386a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:18.728585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8abca-66f7-4822-bd60-d49460386a9c None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:18.729265 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8abca-66f7-4822-bd60-d49460386a9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:18.729567 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8abca-66f7-4822-bd60-d49460386a9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:18.730986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd8abca-66f7-4822-bd60-d49460386a9c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:18.743286 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 629/1272] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:55:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:18.749373 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c864abd-2ee0-4a92-9ed4-7e3888329648 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:18.837776 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:18.846868 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c864abd-2ee0-4a92-9ed4-7e3888329648 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:18.880612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb06a19-7358-441d-a040-95f8ac1c2db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:18.881303 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb06a19-7358-441d-a040-95f8ac1c2db2 None admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:18.881586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb06a19-7358-441d-a040-95f8ac1c2db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:18.881863 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb06a19-7358-441d-a040-95f8ac1c2db2 None admin] PATH_INFO: `/v3/projects/fda69e8c2d5e466dbc1244b2f76296a3/users/18798838de284c84b5b838b3530a2c43/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:18.883760 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb06a19-7358-441d-a040-95f8ac1c2db2 None admin] RBAC: Authorizing `identity:create_grant(project_id=fda69e8c2d5e466dbc1244b2f76296a3, user_id=18798838de284c84b5b838b3530a2c43, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:18.885209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb06a19-7358-441d-a040-95f8ac1c2db2 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:18.898767 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 644/1273] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 00:55:18 2022] PUT /identity/v3/projects/fda69e8c2d5e466dbc1244b2f76296a3/users/18798838de284c84b5b838b3530a2c43/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 00:55:21.256522 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f4b09-b35f-4e05-940c-15f37c29acb3 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:21.257090 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f4b09-b35f-4e05-940c-15f37c29acb3 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:21.257090 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f4b09-b35f-4e05-940c-15f37c29acb3 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:21.258590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 630/1274] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:55:21.264031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c5518-b3a6-4c75-a4bc-718959df3b94 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:21.264475 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c5518-b3a6-4c75-a4bc-718959df3b94 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:21.264576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c5518-b3a6-4c75-a4bc-718959df3b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:21.283113 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f52c5518-b3a6-4c75-a4bc-718959df3b94 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:55:21.284333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f52c5518-b3a6-4c75-a4bc-718959df3b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:55:21.349170 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 645/1275] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:55:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:55:21.618999 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f4b09-b35f-4e05-940c-15f37c29acb3 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:21.717797 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f4b09-b35f-4e05-940c-15f37c29acb3 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:21.734331 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:21.751103 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551c8f5-b97a-42c3-b72a-87b3a421cf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:21.751752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551c8f5-b97a-42c3-b72a-87b3a421cf5a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:21.752029 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551c8f5-b97a-42c3-b72a-87b3a421cf5a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:21.752301 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551c8f5-b97a-42c3-b72a-87b3a421cf5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:21.763048 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551c8f5-b97a-42c3-b72a-87b3a421cf5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 00:55:21.767765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-4551c8f5-b97a-42c3-b72a-87b3a421cf5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:55:21.767765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:21.767765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:21.767765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:21.767765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:21.767765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:21.767765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:21.767765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:21.767765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:21.767765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:21.767765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:21.767765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:21.767765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 00:55:21.767765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 00:55:21.767765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 00:55:21.767765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 00:55:21.768863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 00:55:21.768863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 00:55:21.768863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 00:55:21.768863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 00:55:21.768863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:21.768863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:21.768863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:21.768863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:21.768863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:21.768863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:21.768863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:21.768863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:21.768863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:21.768863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:21.768863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:21.768863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:21.769868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:21.769868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:21.769868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:21.769868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:21.769868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 00:55:21.769868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 00:55:21.769868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 00:55:21.769868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 00:55:21.769868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 00:55:21.769868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 00:55:21.769868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 00:55:21.769868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:55:21.769868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 631/1276] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 00:55:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:21.774902 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c5518-b3a6-4c75-a4bc-718959df3b94 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:21.852814 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:21.871685 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c5518-b3a6-4c75-a4bc-718959df3b94 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:21.904559 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97b666-ffae-43ef-81ba-7b7b9afdc36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:21.905157 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97b666-ffae-43ef-81ba-7b7b9afdc36f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:21.905436 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97b666-ffae-43ef-81ba-7b7b9afdc36f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:21.905709 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97b666-ffae-43ef-81ba-7b7b9afdc36f None admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:21.906361 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97b666-ffae-43ef-81ba-7b7b9afdc36f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:21.906661 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97b666-ffae-43ef-81ba-7b7b9afdc36f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:21.908058 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97b666-ffae-43ef-81ba-7b7b9afdc36f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:21.916368 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 646/1277] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 00:55:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:21.922374 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551c8f5-b97a-42c3-b72a-87b3a421cf5a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:22.019701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551c8f5-b97a-42c3-b72a-87b3a421cf5a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:22.053084 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdb2f2-289d-4826-9d0a-0048ded4400b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:22.053697 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdb2f2-289d-4826-9d0a-0048ded4400b None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:22.053976 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdb2f2-289d-4826-9d0a-0048ded4400b None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:22.054249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdb2f2-289d-4826-9d0a-0048ded4400b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:22.063940 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bdb2f2-289d-4826-9d0a-0048ded4400b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:22.065379 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bdb2f2-289d-4826-9d0a-0048ded4400b None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:22.079398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-25bdb2f2-289d-4826-9d0a-0048ded4400b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:55:22.079398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:22.079398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:22.079398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:22.079398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:22.079398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:22.079398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:22.079398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:22.079398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:22.079398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:22.079398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:22.079398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:22.079398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:55:22.079398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:55:22.079398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:55:22.079398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:55:22.080478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:22.080478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:22.080478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:22.080478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:22.080478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:22.080478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:22.080478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:22.080478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:22.080478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:22.080478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:22.080478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:22.080478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:22.080478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:22.080478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:22.080478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:22.080478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:22.080478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:55:22.081784 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:55:22.081784 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:55:22.081784 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:55:22.081784 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 632/1278] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:55:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:22.086882 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97b666-ffae-43ef-81ba-7b7b9afdc36f None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:22.197145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97b666-ffae-43ef-81ba-7b7b9afdc36f None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:22.230401 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5893e9-7a60-4d26-8050-7b50ca5bbc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:22.231002 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5893e9-7a60-4d26-8050-7b50ca5bbc8b None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:22.231284 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5893e9-7a60-4d26-8050-7b50ca5bbc8b None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:22.231583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5893e9-7a60-4d26-8050-7b50ca5bbc8b None admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:22.232224 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5893e9-7a60-4d26-8050-7b50ca5bbc8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:22.232527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5893e9-7a60-4d26-8050-7b50ca5bbc8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:22.233891 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5893e9-7a60-4d26-8050-7b50ca5bbc8b None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:22.245324 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 647/1279] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:55:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:22.251101 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdb2f2-289d-4826-9d0a-0048ded4400b None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:22.281750 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:22.350382 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdb2f2-289d-4826-9d0a-0048ded4400b None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:22.382721 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442d207-fd7d-4ad4-9b06-48cb87b39c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:22.383337 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442d207-fd7d-4ad4-9b06-48cb87b39c57 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:22.383645 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442d207-fd7d-4ad4-9b06-48cb87b39c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:22.383923 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442d207-fd7d-4ad4-9b06-48cb87b39c57 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:22.391329 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442d207-fd7d-4ad4-9b06-48cb87b39c57 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:22.392807 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442d207-fd7d-4ad4-9b06-48cb87b39c57 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:22.404214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-7442d207-fd7d-4ad4-9b06-48cb87b39c57 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 00:55:22.404214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:22.404214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:22.404214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:22.404214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:22.404214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:22.404214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:22.404214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:22.404214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:22.404214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:22.404214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:22.404214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:22.404214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 00:55:22.404214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 00:55:22.404214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 00:55:22.404214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 00:55:22.405505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:22.405505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:22.405505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 00:55:22.405505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:55:22.405505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 00:55:22.405505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 00:55:22.405505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:22.405505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:22.405505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:22.405505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:22.405505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:22.405505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:22.405505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:22.405505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:22.405505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:22.405505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:22.405505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:22.405505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:22.406815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:22.406815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:22.406815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 00:55:22.406815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 00:55:22.406815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 00:55:22.406815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 00:55:22.406815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 00:55:22.406815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 00:55:22.406815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 00:55:22.406815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:55:22.406815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 633/1280] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:55:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:22.411060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5893e9-7a60-4d26-8050-7b50ca5bbc8b None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:22.414986 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:22.509713 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5893e9-7a60-4d26-8050-7b50ca5bbc8b None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:22.543169 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7ec2b-a794-424b-94cd-b1436b894d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:22.543794 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7ec2b-a794-424b-94cd-b1436b894d12 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:22.544073 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7ec2b-a794-424b-94cd-b1436b894d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:22.544347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7ec2b-a794-424b-94cd-b1436b894d12 None admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:22.545066 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7ec2b-a794-424b-94cd-b1436b894d12 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:22.545370 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7ec2b-a794-424b-94cd-b1436b894d12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:22.546791 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7ec2b-a794-424b-94cd-b1436b894d12 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:22.563023 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 648/1281] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Sep 14 00:55:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:22.569500 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442d207-fd7d-4ad4-9b06-48cb87b39c57 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:22.658300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:22.667927 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442d207-fd7d-4ad4-9b06-48cb87b39c57 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:22.701342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d55504-a02f-449c-9f32-4e10344ef83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:22.701937 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d55504-a02f-449c-9f32-4e10344ef83a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:22.702220 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d55504-a02f-449c-9f32-4e10344ef83a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:22.702495 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d55504-a02f-449c-9f32-4e10344ef83a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:22.767748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d55504-a02f-449c-9f32-4e10344ef83a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:22.769137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d55504-a02f-449c-9f32-4e10344ef83a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:22.854841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:22.870619 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 634/1282] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 00:55:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 14 00:55:22.876496 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7ec2b-a794-424b-94cd-b1436b894d12 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:22.936402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:22.974517 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7ec2b-a794-424b-94cd-b1436b894d12 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:23.008229 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c6468-883b-4dc8-9d37-bfdb4b95e148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:23.008844 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c6468-883b-4dc8-9d37-bfdb4b95e148 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:23.009125 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c6468-883b-4dc8-9d37-bfdb4b95e148 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:23.009396 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c6468-883b-4dc8-9d37-bfdb4b95e148 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:23.019235 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c6468-883b-4dc8-9d37-bfdb4b95e148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:23.020647 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c6468-883b-4dc8-9d37-bfdb4b95e148 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:23.034525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-c19c6468-883b-4dc8-9d37-bfdb4b95e148 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:55:23.034525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:23.034525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:23.034525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:23.034525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:23.034525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:23.034525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:23.034525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:23.034525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:23.034525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:23.034525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:23.034525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:23.034525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 00:55:23.034525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 00:55:23.034525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 00:55:23.034525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 00:55:23.035653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:23.035653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:23.035653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:23.035653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:23.035653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:23.035653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:23.035653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:23.035653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:23.035653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:23.035653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:23.035653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:23.035653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:23.035653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:23.035653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:23.035653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:23.035653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:23.035653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 00:55:23.037300 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 00:55:23.037300 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 00:55:23.037300 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:55:23.039704 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 649/1283] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:55:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:23.041718 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d55504-a02f-449c-9f32-4e10344ef83a None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:23.139176 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d55504-a02f-449c-9f32-4e10344ef83a None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:23.173073 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee6c6e-4eb6-4e56-bf8f-0a13db60c28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:23.173671 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee6c6e-4eb6-4e56-bf8f-0a13db60c28d None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:23.173951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee6c6e-4eb6-4e56-bf8f-0a13db60c28d None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:23.174228 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee6c6e-4eb6-4e56-bf8f-0a13db60c28d None admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:23.174860 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee6c6e-4eb6-4e56-bf8f-0a13db60c28d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:23.175163 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee6c6e-4eb6-4e56-bf8f-0a13db60c28d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:23.176557 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee6c6e-4eb6-4e56-bf8f-0a13db60c28d None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:23.187951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 635/1284] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 00:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:23.193861 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c6468-883b-4dc8-9d37-bfdb4b95e148 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:23.291959 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c6468-883b-4dc8-9d37-bfdb4b95e148 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:23.304966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:23.324969 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55e71e-29a9-4f63-bd70-4c90dbb07af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:23.325584 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55e71e-29a9-4f63-bd70-4c90dbb07af8 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:23.325861 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55e71e-29a9-4f63-bd70-4c90dbb07af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:23.326131 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55e71e-29a9-4f63-bd70-4c90dbb07af8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:23.332423 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55e71e-29a9-4f63-bd70-4c90dbb07af8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:23.334423 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55e71e-29a9-4f63-bd70-4c90dbb07af8 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:23.345194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application [None req-fd55e71e-29a9-4f63-bd70-4c90dbb07af8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:55:23.345194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:23.345194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:23.345194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:23.345194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:23.345194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:23.345194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:23.345194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:23.345194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:23.345194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:23.345194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:23.345194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:23.345194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 00:55:23.345194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 00:55:23.345194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 00:55:23.346247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 00:55:23.346247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:23.346247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:23.346247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:23.346247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:23.346247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:23.346247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:23.346247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:23.346247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:23.346247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:23.346247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:23.346247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:23.346247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:23.346247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:23.346247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:23.346247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:23.346247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:23.347365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 00:55:23.347365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 00:55:23.347365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 00:55:23.347365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 00:55:23.347365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 00:55:23.347365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 00:55:23.347365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 00:55:23.347365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: ERROR keystone.server.flask.application Sep 14 00:55:23.347365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 650/1285] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 00:55:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:23.352548 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee6c6e-4eb6-4e56-bf8f-0a13db60c28d None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:23.401816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:23.451258 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee6c6e-4eb6-4e56-bf8f-0a13db60c28d None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:23.484773 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e86010-dd82-41e5-be16-115b25b826be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:23.485386 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e86010-dd82-41e5-be16-115b25b826be None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:23.485668 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e86010-dd82-41e5-be16-115b25b826be None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:23.485942 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e86010-dd82-41e5-be16-115b25b826be None admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:23.486636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e86010-dd82-41e5-be16-115b25b826be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:23.486942 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e86010-dd82-41e5-be16-115b25b826be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:23.488470 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e86010-dd82-41e5-be16-115b25b826be None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:23.500890 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 636/1286] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 00:55:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:23.507522 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55e71e-29a9-4f63-bd70-4c90dbb07af8 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:23.604528 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55e71e-29a9-4f63-bd70-4c90dbb07af8 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:23.638707 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331d05f-4076-401e-9d95-d979f53367f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:23.639306 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331d05f-4076-401e-9d95-d979f53367f0 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:23.639613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331d05f-4076-401e-9d95-d979f53367f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:23.639890 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331d05f-4076-401e-9d95-d979f53367f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:23.640610 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d331d05f-4076-401e-9d95-d979f53367f0 None admin] RBAC: Adding query filter params (role.id=ada268ea63354a0297070369dfe3c6cc, scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3, user.id=18798838de284c84b5b838b3530a2c43) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:23.640921 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d331d05f-4076-401e-9d95-d979f53367f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:23.642356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d331d05f-4076-401e-9d95-d979f53367f0 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:23.648285 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 651/1287] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 00:55:23 2022] GET /identity/v3/role_assignments?user.id=18798838de284c84b5b838b3530a2c43&scope.project.id=fda69e8c2d5e466dbc1244b2f76296a3&role.id=ada268ea63354a0297070369dfe3c6cc => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:25.808243 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc927c1-c629-4ff1-8467-70f2a47b2079 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:25.808756 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc927c1-c629-4ff1-8467-70f2a47b2079 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:25.808756 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc927c1-c629-4ff1-8467-70f2a47b2079 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:25.810028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 637/1288] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:55:25.815129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c939b9-84ed-4b07-b948-b671529e8d29 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:25.815540 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c939b9-84ed-4b07-b948-b671529e8d29 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:25.815717 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c939b9-84ed-4b07-b948-b671529e8d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:25.834161 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-82c939b9-84ed-4b07-b948-b671529e8d29 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:55:25.835321 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-82c939b9-84ed-4b07-b948-b671529e8d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:55:25.863816 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:25.900896 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 652/1289] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:55:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:55:26.177701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc927c1-c629-4ff1-8467-70f2a47b2079 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:26.281037 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc927c1-c629-4ff1-8467-70f2a47b2079 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:26.291483 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:26.314684 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5b126-978f-442a-aa90-de63f7be05cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:26.315310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5b126-978f-442a-aa90-de63f7be05cf None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:26.315621 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5b126-978f-442a-aa90-de63f7be05cf None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:26.315899 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5b126-978f-442a-aa90-de63f7be05cf None admin] PATH_INFO: `/v3/services/placement` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:26.316479 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5b126-978f-442a-aa90-de63f7be05cf None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:26.318410 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5b126-978f-442a-aa90-de63f7be05cf None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:26.327183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-a7c5b126-978f-442a-aa90-de63f7be05cf None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 00:55:26.327183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:26.327183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:55:26.327183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:55:26.327183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:55:26.327183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:55:26.327183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:55:26.327183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:55:26.327183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 00:55:26.327183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:55:26.327183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:55:26.327183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:55:26.327183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:26.327183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:26.327183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:26.327183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:26.327183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:26.328470 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:26.328470 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:26.328470 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:26.328470 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:26.328470 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:26.328470 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:26.328470 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:55:26.328470 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:55:26.328470 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:55:26.328470 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:55:26.328470 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:26.328470 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:26.328470 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:26.328470 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:26.328470 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:26.328470 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:26.329702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:26.329702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:26.329702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:26.329702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:26.329702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:26.329702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:26.329702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:26.329702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:26.329702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:26.329702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:26.329702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:55:26.329702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:55:26.329702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 00:55:26.329702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:55:26.329702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 638/1290] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Sep 14 00:55:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:26.334923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c939b9-84ed-4b07-b948-b671529e8d29 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:26.430925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c939b9-84ed-4b07-b948-b671529e8d29 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:26.463624 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb921f4-2418-4b8b-a931-40d2e057332f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:26.464226 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb921f4-2418-4b8b-a931-40d2e057332f None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:26.464504 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb921f4-2418-4b8b-a931-40d2e057332f None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:26.464774 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb921f4-2418-4b8b-a931-40d2e057332f None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:26.465393 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb921f4-2418-4b8b-a931-40d2e057332f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:26.465697 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb921f4-2418-4b8b-a931-40d2e057332f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:26.467060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb921f4-2418-4b8b-a931-40d2e057332f None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:26.472005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 653/1291] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Sep 14 00:55:26 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:26.477778 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5b126-978f-442a-aa90-de63f7be05cf None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:26.577416 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5b126-978f-442a-aa90-de63f7be05cf None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:26.609505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1166e8-a936-46ba-aee2-ae36481c665a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:26.610101 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1166e8-a936-46ba-aee2-ae36481c665a None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:26.610382 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1166e8-a936-46ba-aee2-ae36481c665a None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:26.610653 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1166e8-a936-46ba-aee2-ae36481c665a None admin] PATH_INFO: `/v3/services` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:26.611271 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1166e8-a936-46ba-aee2-ae36481c665a None admin] RBAC: Adding query filter params (type=placement) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:26.611591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1166e8-a936-46ba-aee2-ae36481c665a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:26.612959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1166e8-a936-46ba-aee2-ae36481c665a None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:26.617921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 639/1292] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Sep 14 00:55:26 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:28.743758 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d89c1d-fa21-441a-bfc9-2cf2e98a8573 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:28.744282 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d89c1d-fa21-441a-bfc9-2cf2e98a8573 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:28.744282 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d89c1d-fa21-441a-bfc9-2cf2e98a8573 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:28.745561 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 654/1293] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:55:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:55:28.750773 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2987f2c-e0b2-4314-ba88-bc605b16edb3 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:28.751169 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2987f2c-e0b2-4314-ba88-bc605b16edb3 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:28.751314 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2987f2c-e0b2-4314-ba88-bc605b16edb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:28.769583 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-f2987f2c-e0b2-4314-ba88-bc605b16edb3 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:55:28.771236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f2987f2c-e0b2-4314-ba88-bc605b16edb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:55:28.800623 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:28.837299 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 640/1294] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:55:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:55:29.111837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d89c1d-fa21-441a-bfc9-2cf2e98a8573 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:29.148559 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:29.230775 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d89c1d-fa21-441a-bfc9-2cf2e98a8573 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:29.264011 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a96670-12e7-4f06-9293-ef4ad34910f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:29.264610 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a96670-12e7-4f06-9293-ef4ad34910f6 None admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:29.264891 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a96670-12e7-4f06-9293-ef4ad34910f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:29.265159 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a96670-12e7-4f06-9293-ef4ad34910f6 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:29.265796 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a96670-12e7-4f06-9293-ef4ad34910f6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:29.267161 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a96670-12e7-4f06-9293-ef4ad34910f6 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:29.281534 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 655/1295] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 00:55:29 2022] POST /identity/v3/services => generated 246 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:55:31.405354 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf6d73-a938-44f9-bf68-18c157468b92 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:31.405894 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf6d73-a938-44f9-bf68-18c157468b92 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:31.405894 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf6d73-a938-44f9-bf68-18c157468b92 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:31.407231 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 641/1296] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:55:31.413263 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599eb839-65cd-4eb1-b60f-dbd39e31b01d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:31.413712 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599eb839-65cd-4eb1-b60f-dbd39e31b01d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:31.413819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599eb839-65cd-4eb1-b60f-dbd39e31b01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:31.432336 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-599eb839-65cd-4eb1-b60f-dbd39e31b01d None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:55:31.433503 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-599eb839-65cd-4eb1-b60f-dbd39e31b01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:55:31.506851 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 656/1297] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:55:31 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:55:31.773623 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf6d73-a938-44f9-bf68-18c157468b92 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:31.871433 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf6d73-a938-44f9-bf68-18c157468b92 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:31.904493 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a06cbe-7f7c-49f9-b99b-bc3162322d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:31.905104 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a06cbe-7f7c-49f9-b99b-bc3162322d5e None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:31.905382 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a06cbe-7f7c-49f9-b99b-bc3162322d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:31.905653 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a06cbe-7f7c-49f9-b99b-bc3162322d5e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:31.906220 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a06cbe-7f7c-49f9-b99b-bc3162322d5e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:31.907598 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a06cbe-7f7c-49f9-b99b-bc3162322d5e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:31.916711 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-30a06cbe-7f7c-49f9-b99b-bc3162322d5e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 00:55:31.916711 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:31.916711 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:55:31.916711 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:55:31.916711 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:55:31.916711 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:55:31.916711 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:55:31.916711 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:55:31.916711 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 00:55:31.916711 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:55:31.916711 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:55:31.916711 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:55:31.916711 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:31.916711 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:31.916711 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:31.916711 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:31.916711 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:31.917948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:31.917948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:31.917948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:31.917948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:31.917948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:31.917948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:31.917948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:55:31.917948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:55:31.917948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:55:31.917948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:55:31.917948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:31.917948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:31.917948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:31.917948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:31.917948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:31.917948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:31.919016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:31.919016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:31.919016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:31.919016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:31.919016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:31.919016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:31.919016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:31.919016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:31.919016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:31.919016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:31.919016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:55:31.919016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:55:31.919016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 00:55:31.919016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:55:31.919016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 642/1298] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Sep 14 00:55:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:31.923832 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599eb839-65cd-4eb1-b60f-dbd39e31b01d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:31.941576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:32.021291 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599eb839-65cd-4eb1-b60f-dbd39e31b01d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:32.054452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4904dc18-5cf9-4889-8450-09615f20b489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:32.054992 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904dc18-5cf9-4889-8450-09615f20b489 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:32.055274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904dc18-5cf9-4889-8450-09615f20b489 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:32.055548 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904dc18-5cf9-4889-8450-09615f20b489 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:32.056202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4904dc18-5cf9-4889-8450-09615f20b489 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:32.056503 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4904dc18-5cf9-4889-8450-09615f20b489 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:32.057862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4904dc18-5cf9-4889-8450-09615f20b489 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:32.062822 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 657/1299] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Sep 14 00:55:31 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:32.069466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a06cbe-7f7c-49f9-b99b-bc3162322d5e None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:32.101437 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:32.169256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a06cbe-7f7c-49f9-b99b-bc3162322d5e None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:32.201829 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b765516-7683-4812-abc8-bc757b7d7b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:32.202420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b765516-7683-4812-abc8-bc757b7d7b0c None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:32.202695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b765516-7683-4812-abc8-bc757b7d7b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:32.202963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b765516-7683-4812-abc8-bc757b7d7b0c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:32.203672 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b765516-7683-4812-abc8-bc757b7d7b0c None admin] RBAC: Adding query filter params (interface=public, service_id=bfbdd3fe4b8f475d89860f2052c31143, region_id=RegionOne) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:32.203983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b765516-7683-4812-abc8-bc757b7d7b0c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:32.205332 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b765516-7683-4812-abc8-bc757b7d7b0c None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:32.210396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 643/1300] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Sep 14 00:55:32 2022] GET /identity/v3/endpoints?service_id=bfbdd3fe4b8f475d89860f2052c31143&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:32.216104 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4904dc18-5cf9-4889-8450-09615f20b489 None admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:32.295006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:32.312664 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4904dc18-5cf9-4889-8450-09615f20b489 None admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:32.345504 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd44b04-d5eb-47ea-877b-3de4e6a54af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:32.346100 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd44b04-d5eb-47ea-877b-3de4e6a54af7 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:32.346382 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd44b04-d5eb-47ea-877b-3de4e6a54af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:32.346654 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd44b04-d5eb-47ea-877b-3de4e6a54af7 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:32.347249 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd44b04-d5eb-47ea-877b-3de4e6a54af7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:32.349177 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd44b04-d5eb-47ea-877b-3de4e6a54af7 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:32.355906 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 658/1301] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Sep 14 00:55:32 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 00:55:34.486184 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f223d-e2da-4d06-bb51-3658f521a3b5 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:34.486577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f223d-e2da-4d06-bb51-3658f521a3b5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:34.486745 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f223d-e2da-4d06-bb51-3658f521a3b5 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:34.488117 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 644/1302] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:55:34.493184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb24a6-12b0-4349-9cf6-d1276bbb9aa2 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:34.493577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb24a6-12b0-4349-9cf6-d1276bbb9aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:34.493732 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb24a6-12b0-4349-9cf6-d1276bbb9aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:34.512602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4dbb24a6-12b0-4349-9cf6-d1276bbb9aa2 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:55:34.513780 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4dbb24a6-12b0-4349-9cf6-d1276bbb9aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:55:34.580163 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 659/1303] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:55:34 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 00:55:34.847525 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f223d-e2da-4d06-bb51-3658f521a3b5 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:34.940239 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:34.947391 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f223d-e2da-4d06-bb51-3658f521a3b5 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:34.980872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaadb26-baa4-4701-902a-c580883bc4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:34.981476 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaadb26-baa4-4701-902a-c580883bc4c9 None admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:34.981755 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaadb26-baa4-4701-902a-c580883bc4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:34.982026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaadb26-baa4-4701-902a-c580883bc4c9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:34.982598 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaadb26-baa4-4701-902a-c580883bc4c9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:34.984005 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaadb26-baa4-4701-902a-c580883bc4c9 None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:34.992891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application [None req-bdaadb26-baa4-4701-902a-c580883bc4c9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 00:55:34.992891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:34.992891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 00:55:34.992891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 00:55:34.992891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 00:55:34.992891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 00:55:34.992891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 00:55:34.992891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:55:34.992891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 00:55:34.992891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:55:34.992891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 00:55:34.992891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:55:34.992891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 00:55:34.992891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 00:55:34.992891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 00:55:34.992891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 00:55:34.992891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 00:55:34.994098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 00:55:34.994098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 00:55:34.994098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 00:55:34.994098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 00:55:34.994098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 00:55:34.994098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 00:55:34.994098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 00:55:34.994098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 00:55:34.994098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 00:55:34.994098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 00:55:34.994098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 00:55:34.994098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 00:55:34.994098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 00:55:34.994098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 00:55:34.994098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 00:55:34.994098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 00:55:34.995363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 00:55:34.995363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application with Lock( Sep 14 00:55:34.995363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 00:55:34.995363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self._enter() Sep 14 00:55:34.995363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 00:55:34.995363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 00:55:34.995363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 00:55:34.995363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application return self.creator() Sep 14 00:55:34.995363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 00:55:34.995363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application created_value = creator( Sep 14 00:55:34.995363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 00:55:34.995363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 00:55:34.995363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 00:55:34.995363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: ERROR keystone.server.flask.application Sep 14 00:55:34.995363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 645/1304] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Sep 14 00:55:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 00:55:34.999993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb24a6-12b0-4349-9cf6-d1276bbb9aa2 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:35.074033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:35.096735 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb24a6-12b0-4349-9cf6-d1276bbb9aa2 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:35.128828 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c2940-8005-446d-a7fd-7d866079b8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:35.129371 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c2940-8005-446d-a7fd-7d866079b8c9 None admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:35.130212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c2940-8005-446d-a7fd-7d866079b8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:35.130492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c2940-8005-446d-a7fd-7d866079b8c9 None admin] PATH_INFO: `/v3/services` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:35.131117 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c2940-8005-446d-a7fd-7d866079b8c9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 00:55:35.131421 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c2940-8005-446d-a7fd-7d866079b8c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:35.132816 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c2940-8005-446d-a7fd-7d866079b8c9 None admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:35.138144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 660/1305] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Sep 14 00:55:34 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 00:55:35.143843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaadb26-baa4-4701-902a-c580883bc4c9 None admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:55:35.241522 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaadb26-baa4-4701-902a-c580883bc4c9 None admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:55:35.274720 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dbfd85-033e-49a6-a985-a9a55937403e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:55:35.275311 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dbfd85-033e-49a6-a985-a9a55937403e None admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:55:35.275616 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dbfd85-033e-49a6-a985-a9a55937403e None admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:55:35.275896 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dbfd85-033e-49a6-a985-a9a55937403e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:55:35.276549 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dbfd85-033e-49a6-a985-a9a55937403e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:55:35.277920 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dbfd85-033e-49a6-a985-a9a55937403e None admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:55:35.297131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 646/1306] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 00:55:35 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 00:55:45.134917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=30 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:55:45.292974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:56:38.999258 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722ee54-c9ec-45a5-8a71-91acf4f1fbd3 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:56:38.999782 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722ee54-c9ec-45a5-8a71-91acf4f1fbd3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:56:38.999885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722ee54-c9ec-45a5-8a71-91acf4f1fbd3 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:56:39.005164 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 661/1307] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:56:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:56:39.006858 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412d8b2-6d51-494c-9591-c8e281c3933d None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:56:39.008147 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412d8b2-6d51-494c-9591-c8e281c3933d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:56:39.009356 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412d8b2-6d51-494c-9591-c8e281c3933d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:56:39.118903 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-b412d8b2-6d51-494c-9591-c8e281c3933d None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:56:39.120553 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b412d8b2-6d51-494c-9591-c8e281c3933d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:56:39.248206 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 647/1308] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:56:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 00:56:39.555068 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ee50d-8664-4ae7-9909-434906af7716 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:56:39.555256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ee50d-8664-4ae7-9909-434906af7716 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:56:39.555535 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ee50d-8664-4ae7-9909-434906af7716 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:56:39.557145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 662/1309] 10.222.0.42 () {54 vars in 965 bytes} [Wed Sep 14 00:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:56:39.563975 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbb2e9-f4c4-4d33-855a-59a5e8a43adf None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:56:39.564520 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbb2e9-f4c4-4d33-855a-59a5e8a43adf None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:56:39.565028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbb2e9-f4c4-4d33-855a-59a5e8a43adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:56:39.653877 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-90dbb2e9-f4c4-4d33-855a-59a5e8a43adf None None] MFA Rules not processed for user `4d64f9bca7c546e4ba7f54c1c47d665e`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:56:39.655387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-90dbb2e9-f4c4-4d33-855a-59a5e8a43adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:56:39.692694 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:56:39.769146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ee50d-8664-4ae7-9909-434906af7716 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:56:39.770908 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b12ee50d-8664-4ae7-9909-434906af7716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:56:39.775595 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 648/1310] 10.222.0.42 () {58 vars in 1062 bytes} [Wed Sep 14 00:56:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:56:39.910199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ee50d-8664-4ae7-9909-434906af7716 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:56:39.996242 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03847a02-ba64-45dd-91d3-aeb549d86143 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:56:39.997041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03847a02-ba64-45dd-91d3-aeb549d86143 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:56:39.997419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03847a02-ba64-45dd-91d3-aeb549d86143 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:56:39.997794 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03847a02-ba64-45dd-91d3-aeb549d86143 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:56:39.999510 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-03847a02-ba64-45dd-91d3-aeb549d86143 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:56:40.060407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:56:40.123875 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03847a02-ba64-45dd-91d3-aeb549d86143 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:56:40.125635 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03847a02-ba64-45dd-91d3-aeb549d86143 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:56:40.147184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 663/1311] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:56:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 00:56:42.722885 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628c26b-18a0-4585-9366-959ac744bd38 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:56:42.723451 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628c26b-18a0-4585-9366-959ac744bd38 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:56:42.723451 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628c26b-18a0-4585-9366-959ac744bd38 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:56:42.724826 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 649/1312] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:56:42.730038 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0512915-5181-4e45-b9fe-5023f0f75d01 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:56:42.730465 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0512915-5181-4e45-b9fe-5023f0f75d01 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:56:42.730575 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0512915-5181-4e45-b9fe-5023f0f75d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:56:42.813691 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d0512915-5181-4e45-b9fe-5023f0f75d01 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:56:42.814843 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d0512915-5181-4e45-b9fe-5023f0f75d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:56:42.910080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 664/1313] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:56:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:56:43.189258 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628c26b-18a0-4585-9366-959ac744bd38 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:56:43.190795 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1628c26b-18a0-4585-9366-959ac744bd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:56:43.294487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628c26b-18a0-4585-9366-959ac744bd38 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:56:43.365958 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:56:43.368706 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9376553-7ccd-41aa-96ac-380b61d2c02d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:56:43.369545 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9376553-7ccd-41aa-96ac-380b61d2c02d service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:56:43.370002 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9376553-7ccd-41aa-96ac-380b61d2c02d service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:56:43.370320 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9376553-7ccd-41aa-96ac-380b61d2c02d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:56:43.372112 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b9376553-7ccd-41aa-96ac-380b61d2c02d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:56:43.480897 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9376553-7ccd-41aa-96ac-380b61d2c02d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:56:43.481997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9376553-7ccd-41aa-96ac-380b61d2c02d service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:56:43.500819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 650/1314] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:56:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 00:56:45.954081 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0764d1-7fab-496e-8ea3-798c81f00559 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:56:45.954635 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0764d1-7fab-496e-8ea3-798c81f00559 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:56:45.954635 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0764d1-7fab-496e-8ea3-798c81f00559 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:56:45.956024 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 665/1315] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:56:45.961365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1328b842-0984-449b-830a-d3beaab17eed None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:56:45.961782 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1328b842-0984-449b-830a-d3beaab17eed None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:56:45.961919 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1328b842-0984-449b-830a-d3beaab17eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:56:46.028585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-1328b842-0984-449b-830a-d3beaab17eed None None] MFA Rules not processed for user `938f59e04b4b4ad7bb6b69e44e819036`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:56:46.029788 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1328b842-0984-449b-830a-d3beaab17eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:56:46.081651 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:56:46.127495 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 651/1316] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:56:45 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:56:46.413182 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0764d1-7fab-496e-8ea3-798c81f00559 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:56:46.414743 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1e0764d1-7fab-496e-8ea3-798c81f00559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:56:46.424625 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:56:46.520369 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0764d1-7fab-496e-8ea3-798c81f00559 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:56:46.593608 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25fa5f4-385d-4842-83ba-fa6ad67f6c00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:56:46.594309 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fa5f4-385d-4842-83ba-fa6ad67f6c00 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:56:46.594588 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fa5f4-385d-4842-83ba-fa6ad67f6c00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:56:46.594856 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fa5f4-385d-4842-83ba-fa6ad67f6c00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:56:46.596477 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a25fa5f4-385d-4842-83ba-fa6ad67f6c00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:56:46.658229 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:56:46.704957 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25fa5f4-385d-4842-83ba-fa6ad67f6c00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:56:46.706399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25fa5f4-385d-4842-83ba-fa6ad67f6c00 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:56:46.725708 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 666/1317] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:56:46 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:56:49.416680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdb98a-2afd-484a-9dc2-9245f4e25cba None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:56:49.417261 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdb98a-2afd-484a-9dc2-9245f4e25cba None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:56:49.417261 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdb98a-2afd-484a-9dc2-9245f4e25cba None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:56:49.418575 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 652/1318] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:56:49.423914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea54722-09be-432c-8da9-5f18088bdf63 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:56:49.424374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea54722-09be-432c-8da9-5f18088bdf63 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:56:49.424489 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea54722-09be-432c-8da9-5f18088bdf63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:56:49.490631 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4ea54722-09be-432c-8da9-5f18088bdf63 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:56:49.491858 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4ea54722-09be-432c-8da9-5f18088bdf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:56:49.586169 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 667/1319] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:56:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 00:56:49.874945 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdb98a-2afd-484a-9dc2-9245f4e25cba None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:56:49.877181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-07bdb98a-2afd-484a-9dc2-9245f4e25cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:56:49.978392 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdb98a-2afd-484a-9dc2-9245f4e25cba None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:56:50.054132 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06dd3a9-341e-4ceb-9e51-6154dfec9323 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:56:50.054739 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dd3a9-341e-4ceb-9e51-6154dfec9323 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:56:50.055021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dd3a9-341e-4ceb-9e51-6154dfec9323 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:56:50.055298 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dd3a9-341e-4ceb-9e51-6154dfec9323 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:56:50.056917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b06dd3a9-341e-4ceb-9e51-6154dfec9323 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:56:50.063905 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:56:50.167481 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06dd3a9-341e-4ceb-9e51-6154dfec9323 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:56:50.168966 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06dd3a9-341e-4ceb-9e51-6154dfec9323 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:56:50.187038 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 653/1320] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:56:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:56:53.014971 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c3fe7-e6e0-4c74-bdd6-5f5098759018 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:56:53.015619 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c3fe7-e6e0-4c74-bdd6-5f5098759018 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:56:53.015755 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c3fe7-e6e0-4c74-bdd6-5f5098759018 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:56:53.017404 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 668/1321] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:56:53.022835 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c83f9-34d0-4513-8879-b93c1cdecb69 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:56:53.023449 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c83f9-34d0-4513-8879-b93c1cdecb69 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:56:53.023449 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c83f9-34d0-4513-8879-b93c1cdecb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:56:53.091680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-b82c83f9-34d0-4513-8879-b93c1cdecb69 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:56:53.092898 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b82c83f9-34d0-4513-8879-b93c1cdecb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:56:53.158900 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:56:53.202898 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 654/1322] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 00:56:53.505133 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4eec5-58fc-410d-8310-e12ebed49a9d None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:56:53.505424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4eec5-58fc-410d-8310-e12ebed49a9d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:56:53.505701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4eec5-58fc-410d-8310-e12ebed49a9d None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:56:53.507043 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 669/1323] 10.222.0.42 () {54 vars in 965 bytes} [Wed Sep 14 00:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:56:53.513232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcab82b-e2d4-47fb-b2ab-4606a6d80525 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:56:53.513520 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcab82b-e2d4-47fb-b2ab-4606a6d80525 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:56:53.513792 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcab82b-e2d4-47fb-b2ab-4606a6d80525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:56:53.589880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-9dcab82b-e2d4-47fb-b2ab-4606a6d80525 None None] MFA Rules not processed for user `4d64f9bca7c546e4ba7f54c1c47d665e`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:56:53.590968 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9dcab82b-e2d4-47fb-b2ab-4606a6d80525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:56:53.668573 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 655/1324] 10.222.0.42 () {58 vars in 1062 bytes} [Wed Sep 14 00:56:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:56:53.677755 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4eec5-58fc-410d-8310-e12ebed49a9d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:56:53.679377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e1b4eec5-58fc-410d-8310-e12ebed49a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:56:53.698784 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:56:53.784523 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4eec5-58fc-410d-8310-e12ebed49a9d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:56:53.856527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c140a96-a05e-4bcf-8815-a6dcf5af70d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:56:53.857052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c140a96-a05e-4bcf-8815-a6dcf5af70d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:56:53.857332 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c140a96-a05e-4bcf-8815-a6dcf5af70d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:56:53.857600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c140a96-a05e-4bcf-8815-a6dcf5af70d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:56:53.859208 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2c140a96-a05e-4bcf-8815-a6dcf5af70d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:56:53.930517 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:56:53.969528 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c140a96-a05e-4bcf-8815-a6dcf5af70d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:56:53.970979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c140a96-a05e-4bcf-8815-a6dcf5af70d1 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:56:53.988822 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 670/1325] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:56:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:56:56.435907 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af98082-eb92-4bdd-b9e3-e1422fdaaceb None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:56:56.436456 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af98082-eb92-4bdd-b9e3-e1422fdaaceb None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:56:56.436456 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af98082-eb92-4bdd-b9e3-e1422fdaaceb None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:56:56.438062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 656/1326] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:56:56.443363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77c2b9-8ae6-4044-ae11-a9d5ab35cca2 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:56:56.443832 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77c2b9-8ae6-4044-ae11-a9d5ab35cca2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:56:56.443961 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77c2b9-8ae6-4044-ae11-a9d5ab35cca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:56:56.513706 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-9c77c2b9-8ae6-4044-ae11-a9d5ab35cca2 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:56:56.514797 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9c77c2b9-8ae6-4044-ae11-a9d5ab35cca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:56:56.617077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 671/1327] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:56:56.900944 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af98082-eb92-4bdd-b9e3-e1422fdaaceb None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:56:56.902467 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9af98082-eb92-4bdd-b9e3-e1422fdaaceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:56:56.963370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:56:57.008527 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af98082-eb92-4bdd-b9e3-e1422fdaaceb None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:56:57.082536 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c1e90-b9ba-4236-8a1d-ed99ca353fcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:56:57.083151 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c1e90-b9ba-4236-8a1d-ed99ca353fcd service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:56:57.083429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c1e90-b9ba-4236-8a1d-ed99ca353fcd service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:56:57.083731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c1e90-b9ba-4236-8a1d-ed99ca353fcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:56:57.085325 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e30c1e90-b9ba-4236-8a1d-ed99ca353fcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:56:57.197117 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:56:57.199508 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c1e90-b9ba-4236-8a1d-ed99ca353fcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:56:57.201167 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c1e90-b9ba-4236-8a1d-ed99ca353fcd service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:56:57.220357 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 657/1328] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:56:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:56:59.482186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ff2a9-b66f-484c-aa0b-5bd9a191dc27 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:56:59.482564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ff2a9-b66f-484c-aa0b-5bd9a191dc27 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:56:59.482737 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ff2a9-b66f-484c-aa0b-5bd9a191dc27 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:56:59.487935 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 672/1329] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:56:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:56:59.490840 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b2540-9883-487f-b6aa-2365cd292cde None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:56:59.491309 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b2540-9883-487f-b6aa-2365cd292cde None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:56:59.491404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b2540-9883-487f-b6aa-2365cd292cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:56:59.562880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-606b2540-9883-487f-b6aa-2365cd292cde None None] MFA Rules not processed for user `938f59e04b4b4ad7bb6b69e44e819036`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:56:59.564407 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-606b2540-9883-487f-b6aa-2365cd292cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:56:59.666310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 658/1330] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:56:59 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:56:59.954477 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ff2a9-b66f-484c-aa0b-5bd9a191dc27 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:56:59.956099 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-467ff2a9-b66f-484c-aa0b-5bd9a191dc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:56:59.987206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:00.063171 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ff2a9-b66f-484c-aa0b-5bd9a191dc27 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:57:00.138370 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04890bc6-fb3c-457b-80ed-5d791b10848f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:57:00.139307 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04890bc6-fb3c-457b-80ed-5d791b10848f service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:00.139307 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04890bc6-fb3c-457b-80ed-5d791b10848f service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:00.139515 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04890bc6-fb3c-457b-80ed-5d791b10848f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:00.141774 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-04890bc6-fb3c-457b-80ed-5d791b10848f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:00.221254 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:00.254810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04890bc6-fb3c-457b-80ed-5d791b10848f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:57:00.256292 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04890bc6-fb3c-457b-80ed-5d791b10848f service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:57:00.275251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 673/1331] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:56:59 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:57:02.863478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd04b5f-7b88-4221-abeb-7dbe95d26b3a None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:02.864062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd04b5f-7b88-4221-abeb-7dbe95d26b3a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:02.864062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd04b5f-7b88-4221-abeb-7dbe95d26b3a None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:02.865315 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 659/1332] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:57:02.870491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9769bc-13b0-42b8-98d7-ff9f702efd2e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:02.870920 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9769bc-13b0-42b8-98d7-ff9f702efd2e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:02.871055 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9769bc-13b0-42b8-98d7-ff9f702efd2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:02.938204 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2e9769bc-13b0-42b8-98d7-ff9f702efd2e None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:57:02.939432 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2e9769bc-13b0-42b8-98d7-ff9f702efd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:03.034644 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 674/1333] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:57:03.319273 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd04b5f-7b88-4221-abeb-7dbe95d26b3a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:57:03.320910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7cd04b5f-7b88-4221-abeb-7dbe95d26b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:03.390932 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:03.424848 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd04b5f-7b88-4221-abeb-7dbe95d26b3a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:57:03.499037 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b15dd5-79b3-475c-9ddd-0638223bcde6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:57:03.500623 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b15dd5-79b3-475c-9ddd-0638223bcde6 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:03.500903 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b15dd5-79b3-475c-9ddd-0638223bcde6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:03.501175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b15dd5-79b3-475c-9ddd-0638223bcde6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:03.502752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-69b15dd5-79b3-475c-9ddd-0638223bcde6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:03.613021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b15dd5-79b3-475c-9ddd-0638223bcde6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:57:03.614391 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b15dd5-79b3-475c-9ddd-0638223bcde6 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:57:03.624694 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:03.633535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 660/1334] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:57:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:57:05.844376 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b287a-588f-4990-958a-63c6c79e0152 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:05.844950 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b287a-588f-4990-958a-63c6c79e0152 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:05.844950 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b287a-588f-4990-958a-63c6c79e0152 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:05.846317 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 675/1335] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:57:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:57:05.851675 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5478e4f-c062-4e4c-892e-fc3b4cd89a55 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:05.852059 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5478e4f-c062-4e4c-892e-fc3b4cd89a55 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:05.852225 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5478e4f-c062-4e4c-892e-fc3b4cd89a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:05.920655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-c5478e4f-c062-4e4c-892e-fc3b4cd89a55 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:57:05.922486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c5478e4f-c062-4e4c-892e-fc3b4cd89a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:06.016703 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 661/1336] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:57:06.305350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b287a-588f-4990-958a-63c6c79e0152 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:57:06.306918 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-166b287a-588f-4990-958a-63c6c79e0152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:06.347413 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:06.411269 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b287a-588f-4990-958a-63c6c79e0152 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:57:06.486796 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b9fe6-bcb2-4ae1-8de9-26acd929d168 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:57:06.487410 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b9fe6-bcb2-4ae1-8de9-26acd929d168 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:06.487733 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b9fe6-bcb2-4ae1-8de9-26acd929d168 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:06.488011 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b9fe6-bcb2-4ae1-8de9-26acd929d168 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:06.489596 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-847b9fe6-bcb2-4ae1-8de9-26acd929d168 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:06.572261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:06.598570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b9fe6-bcb2-4ae1-8de9-26acd929d168 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:57:06.600046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b9fe6-bcb2-4ae1-8de9-26acd929d168 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:57:06.618183 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 676/1337] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:57:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:57:09.113080 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a03c7-213e-4fa8-bcdd-4122df15185a None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:09.113680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a03c7-213e-4fa8-bcdd-4122df15185a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:09.113680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a03c7-213e-4fa8-bcdd-4122df15185a None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:09.114996 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 662/1338] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:57:09.120866 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bbf3b-4497-49e8-a04c-2effa7eb874f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:09.121330 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bbf3b-4497-49e8-a04c-2effa7eb874f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:09.121620 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bbf3b-4497-49e8-a04c-2effa7eb874f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:09.188670 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-530bbf3b-4497-49e8-a04c-2effa7eb874f None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:57:09.189874 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-530bbf3b-4497-49e8-a04c-2effa7eb874f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:09.288339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 677/1339] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:57:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:57:09.576615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a03c7-213e-4fa8-bcdd-4122df15185a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:57:09.579109 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c09a03c7-213e-4fa8-bcdd-4122df15185a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:09.655201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:09.682525 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a03c7-213e-4fa8-bcdd-4122df15185a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:57:09.755815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c1e87-ccfd-45cc-9195-927008649bd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:57:09.756352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c1e87-ccfd-45cc-9195-927008649bd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:09.756630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c1e87-ccfd-45cc-9195-927008649bd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:09.756902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c1e87-ccfd-45cc-9195-927008649bd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:09.758469 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-fa3c1e87-ccfd-45cc-9195-927008649bd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:09.870277 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c1e87-ccfd-45cc-9195-927008649bd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:57:09.871747 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c1e87-ccfd-45cc-9195-927008649bd2 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:57:09.891100 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 663/1340] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:57:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:57:14.018582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f2baf-05a6-4fc3-8604-78065f28e98a None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:14.019150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f2baf-05a6-4fc3-8604-78065f28e98a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:14.019150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f2baf-05a6-4fc3-8604-78065f28e98a None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:14.020496 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 678/1341] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:57:14.025873 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc198a5-0e33-48bb-9755-b13356743d64 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:14.026275 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc198a5-0e33-48bb-9755-b13356743d64 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:14.026426 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc198a5-0e33-48bb-9755-b13356743d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:14.032805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:14.095023 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-ebc198a5-0e33-48bb-9755-b13356743d64 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:57:14.096189 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ebc198a5-0e33-48bb-9755-b13356743d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:14.193496 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 664/1342] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:57:14.479803 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f2baf-05a6-4fc3-8604-78065f28e98a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:57:14.481510 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-503f2baf-05a6-4fc3-8604-78065f28e98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:14.532928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:14.591544 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f2baf-05a6-4fc3-8604-78065f28e98a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:57:14.667256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752a8d3-3c10-43ff-bafd-cc2425468cf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:57:14.667898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752a8d3-3c10-43ff-bafd-cc2425468cf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:14.668183 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752a8d3-3c10-43ff-bafd-cc2425468cf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:14.668457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752a8d3-3c10-43ff-bafd-cc2425468cf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:14.670039 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3752a8d3-3c10-43ff-bafd-cc2425468cf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:14.763405 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:14.781151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3752a8d3-3c10-43ff-bafd-cc2425468cf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:57:14.782613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3752a8d3-3c10-43ff-bafd-cc2425468cf2 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:57:14.801126 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 679/1343] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:57:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:57:17.637256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40cf0c-ec90-406b-919d-3873aeb2c2af None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:17.637788 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40cf0c-ec90-406b-919d-3873aeb2c2af None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:17.637788 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40cf0c-ec90-406b-919d-3873aeb2c2af None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:17.639771 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 665/1344] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:57:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:57:17.645097 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7fd11-eb5f-4ba6-b3b7-c41d1e321aa9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:17.645557 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7fd11-eb5f-4ba6-b3b7-c41d1e321aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:17.645650 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7fd11-eb5f-4ba6-b3b7-c41d1e321aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:17.711826 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-88c7fd11-eb5f-4ba6-b3b7-c41d1e321aa9 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:57:17.713037 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-88c7fd11-eb5f-4ba6-b3b7-c41d1e321aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:17.814410 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 680/1345] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:57:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:57:18.099291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40cf0c-ec90-406b-919d-3873aeb2c2af None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:57:18.100832 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0f40cf0c-ec90-406b-919d-3873aeb2c2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:18.207200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:18.225324 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40cf0c-ec90-406b-919d-3873aeb2c2af None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:57:18.299792 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4dbc69-2387-459e-a8d6-737f902eb476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:57:18.300313 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dbc69-2387-459e-a8d6-737f902eb476 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:18.300589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dbc69-2387-459e-a8d6-737f902eb476 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:18.300857 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dbc69-2387-459e-a8d6-737f902eb476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:18.302423 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ab4dbc69-2387-459e-a8d6-737f902eb476 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:18.413916 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4dbc69-2387-459e-a8d6-737f902eb476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:57:18.415368 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4dbc69-2387-459e-a8d6-737f902eb476 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:57:18.433086 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 666/1346] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:57:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:57:22.605886 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113042c-df0e-4f67-9089-c47832a055e6 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:22.606355 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113042c-df0e-4f67-9089-c47832a055e6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:22.606519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113042c-df0e-4f67-9089-c47832a055e6 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:22.608808 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 681/1347] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:57:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:57:22.614385 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3ffcf-0567-4973-9af0-29bfcd54b035 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:22.614806 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3ffcf-0567-4973-9af0-29bfcd54b035 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:22.614946 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3ffcf-0567-4973-9af0-29bfcd54b035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:22.631462 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:22.688430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-75f3ffcf-0567-4973-9af0-29bfcd54b035 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:57:22.689651 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-75f3ffcf-0567-4973-9af0-29bfcd54b035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:22.786375 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 667/1348] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:57:23.076094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113042c-df0e-4f67-9089-c47832a055e6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:57:23.077701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5113042c-df0e-4f67-9089-c47832a055e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:23.138959 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:23.193547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113042c-df0e-4f67-9089-c47832a055e6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:57:23.281824 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455d3a9-09d7-425e-bf5f-f4751293359a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:57:23.282438 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455d3a9-09d7-425e-bf5f-f4751293359a service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:23.282723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455d3a9-09d7-425e-bf5f-f4751293359a service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:23.282999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455d3a9-09d7-425e-bf5f-f4751293359a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:23.284642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6455d3a9-09d7-425e-bf5f-f4751293359a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:23.415849 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:23.421038 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455d3a9-09d7-425e-bf5f-f4751293359a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:57:23.422821 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455d3a9-09d7-425e-bf5f-f4751293359a service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:57:23.441948 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 682/1349] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:57:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:57:28.451017 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de588b-53bf-4ca7-aedb-000b382bd4d1 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:28.451672 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de588b-53bf-4ca7-aedb-000b382bd4d1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:28.451873 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de588b-53bf-4ca7-aedb-000b382bd4d1 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:28.453218 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 668/1350] 10.222.0.42 () {54 vars in 921 bytes} [Wed Sep 14 00:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:57:28.458644 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb36f2-a81d-4ae6-8557-16971946f4aa None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:28.459111 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb36f2-a81d-4ae6-8557-16971946f4aa None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:28.459208 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb36f2-a81d-4ae6-8557-16971946f4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:28.609539 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-adeb36f2-a81d-4ae6-8557-16971946f4aa None None] MFA Rules not processed for user `18798838de284c84b5b838b3530a2c43`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:57:28.611618 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-adeb36f2-a81d-4ae6-8557-16971946f4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:28.757864 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 683/1351] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Sep 14 00:57:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:57:32.786263 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=59 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:32.848080 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c632d5-d398-40ca-8ed7-ff284a8be0d8 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:32.848375 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c632d5-d398-40ca-8ed7-ff284a8be0d8 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:32.848840 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c632d5-d398-40ca-8ed7-ff284a8be0d8 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:32.850504 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 669/1352] 10.222.0.42 () {54 vars in 921 bytes} [Wed Sep 14 00:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:57:32.856117 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ab8b9-ce89-4abb-91f9-024f88964e38 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:32.856579 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ab8b9-ce89-4abb-91f9-024f88964e38 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:32.856843 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ab8b9-ce89-4abb-91f9-024f88964e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:32.914594 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:32.939573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-dd8ab8b9-ce89-4abb-91f9-024f88964e38 None None] MFA Rules not processed for user `18798838de284c84b5b838b3530a2c43`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:57:32.940980 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dd8ab8b9-ce89-4abb-91f9-024f88964e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:33.053326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 684/1353] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Sep 14 00:57:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:57:34.489921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4a8bd-0b07-4714-b224-393f2613b2c3 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:34.490404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4a8bd-0b07-4714-b224-393f2613b2c3 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:34.490521 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4a8bd-0b07-4714-b224-393f2613b2c3 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:34.492039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 670/1354] 10.222.0.42 () {54 vars in 921 bytes} [Wed Sep 14 00:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:57:34.496922 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e06017-4776-4a14-9355-8d9040a363ff None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:34.497910 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e06017-4776-4a14-9355-8d9040a363ff None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:34.498148 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e06017-4776-4a14-9355-8d9040a363ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:34.574972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-68e06017-4776-4a14-9355-8d9040a363ff None None] MFA Rules not processed for user `18798838de284c84b5b838b3530a2c43`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:57:34.577136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-68e06017-4776-4a14-9355-8d9040a363ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:34.618058 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:34.684478 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 685/1355] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Sep 14 00:57:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:57:38.281841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5a4d4-94d0-41b6-9c41-6f5017c3b8f8 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:38.282505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5a4d4-94d0-41b6-9c41-6f5017c3b8f8 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:38.282505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5a4d4-94d0-41b6-9c41-6f5017c3b8f8 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:38.283737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 671/1356] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:57:38.289324 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea4400-fe8f-470c-ad79-afa5146c2136 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:38.289805 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea4400-fe8f-470c-ad79-afa5146c2136 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:38.289912 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea4400-fe8f-470c-ad79-afa5146c2136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:38.369031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-6cea4400-fe8f-470c-ad79-afa5146c2136 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:57:38.370860 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6cea4400-fe8f-470c-ad79-afa5146c2136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:38.486904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 686/1357] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:57:38.798384 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf974d-c070-4912-83b8-36822a3dadfa None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:38.798907 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf974d-c070-4912-83b8-36822a3dadfa None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:38.799466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf974d-c070-4912-83b8-36822a3dadfa None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:38.801363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 672/1358] 10.222.0.42 () {54 vars in 962 bytes} [Wed Sep 14 00:57:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:57:38.812350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6a7ca-0aef-49e7-a962-1d963e77df50 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:38.812912 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6a7ca-0aef-49e7-a962-1d963e77df50 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:38.813578 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6a7ca-0aef-49e7-a962-1d963e77df50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:38.832189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:38.895515 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e4b6a7ca-0aef-49e7-a962-1d963e77df50 None None] MFA Rules not processed for user `5e0fe2e84f6748c992c9beedf2f5d49b`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:57:38.896762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e4b6a7ca-0aef-49e7-a962-1d963e77df50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:39.029870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 687/1359] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Sep 14 00:57:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:57:39.033177 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf974d-c070-4912-83b8-36822a3dadfa None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:57:39.034825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0acf974d-c070-4912-83b8-36822a3dadfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:39.194537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf974d-c070-4912-83b8-36822a3dadfa None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:57:39.273993 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:39.322655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa74856-2882-47c3-b1ac-3778d06c2ac5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:57:39.323432 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa74856-2882-47c3-b1ac-3778d06c2ac5 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:39.323791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa74856-2882-47c3-b1ac-3778d06c2ac5 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:39.324073 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa74856-2882-47c3-b1ac-3778d06c2ac5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:39.326025 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-dfa74856-2882-47c3-b1ac-3778d06c2ac5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:39.430463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa74856-2882-47c3-b1ac-3778d06c2ac5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:57:39.432082 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa74856-2882-47c3-b1ac-3778d06c2ac5 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:57:39.452555 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 673/1360] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:57:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 00:57:41.865904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd8e7d-6d61-4e4a-83d9-2042e8577539 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:41.866348 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd8e7d-6d61-4e4a-83d9-2042e8577539 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:41.866696 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd8e7d-6d61-4e4a-83d9-2042e8577539 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:41.873307 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 688/1361] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:57:41.874558 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6d3e8-b08f-4596-8a2b-4fc7257d4464 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:41.874558 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6d3e8-b08f-4596-8a2b-4fc7257d4464 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:41.875008 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6d3e8-b08f-4596-8a2b-4fc7257d4464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:41.946856 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e1f6d3e8-b08f-4596-8a2b-4fc7257d4464 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:57:41.948298 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e1f6d3e8-b08f-4596-8a2b-4fc7257d4464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:41.989994 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:42.053438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 674/1362] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:57:42.344884 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd8e7d-6d61-4e4a-83d9-2042e8577539 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:57:42.346403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c6bd8e7d-6d61-4e4a-83d9-2042e8577539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:42.421477 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:42.540062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd8e7d-6d61-4e4a-83d9-2042e8577539 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:57:42.676694 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3098f3-8acf-452d-b130-5a9117c8086d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:57:42.677301 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3098f3-8acf-452d-b130-5a9117c8086d service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:42.677577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3098f3-8acf-452d-b130-5a9117c8086d service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:42.677852 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3098f3-8acf-452d-b130-5a9117c8086d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:42.679482 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1f3098f3-8acf-452d-b130-5a9117c8086d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:42.703730 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:42.772771 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3098f3-8acf-452d-b130-5a9117c8086d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:57:42.774389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3098f3-8acf-452d-b130-5a9117c8086d service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:57:42.793263 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 689/1363] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:57:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:57:45.317870 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156a734-445c-42a6-a6e4-6b2dcf2f2b4b None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:45.318454 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156a734-445c-42a6-a6e4-6b2dcf2f2b4b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:45.318454 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156a734-445c-42a6-a6e4-6b2dcf2f2b4b None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:45.320946 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 675/1364] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:57:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:57:45.326930 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede0edb-d476-4a48-97ef-665729393f6c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:45.327343 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede0edb-d476-4a48-97ef-665729393f6c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:45.327529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede0edb-d476-4a48-97ef-665729393f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:45.397930 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-cede0edb-d476-4a48-97ef-665729393f6c None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:57:45.399131 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cede0edb-d476-4a48-97ef-665729393f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:45.513317 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:45.515603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 690/1365] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:57:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:57:45.818920 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66ee41-2a57-4181-8391-49a14d0e04b5 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:45.819336 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66ee41-2a57-4181-8391-49a14d0e04b5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:45.819759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66ee41-2a57-4181-8391-49a14d0e04b5 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:45.821222 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 676/1366] 10.222.0.42 () {54 vars in 962 bytes} [Wed Sep 14 00:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:57:45.826781 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47148d5-7aee-4375-8d40-081a76facc5d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:45.827189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47148d5-7aee-4375-8d40-081a76facc5d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:45.827612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47148d5-7aee-4375-8d40-081a76facc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:45.908174 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c47148d5-7aee-4375-8d40-081a76facc5d None None] MFA Rules not processed for user `5e0fe2e84f6748c992c9beedf2f5d49b`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:57:45.909535 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c47148d5-7aee-4375-8d40-081a76facc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:46.039604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 691/1367] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Sep 14 00:57:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 00:57:46.046484 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d66ee41-2a57-4181-8391-49a14d0e04b5 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:57:46.048062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7d66ee41-2a57-4181-8391-49a14d0e04b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:46.206596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d66ee41-2a57-4181-8391-49a14d0e04b5 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:57:46.228834 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:46.333239 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d6656-046d-41b8-ad4f-f44d4d8852e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:57:46.334037 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d6656-046d-41b8-ad4f-f44d4d8852e8 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:46.334416 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d6656-046d-41b8-ad4f-f44d4d8852e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:46.334787 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d6656-046d-41b8-ad4f-f44d4d8852e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:46.336533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c27d6656-046d-41b8-ad4f-f44d4d8852e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:46.439540 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d6656-046d-41b8-ad4f-f44d4d8852e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:57:46.441341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d6656-046d-41b8-ad4f-f44d4d8852e8 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:57:46.461420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 677/1368] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:57:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:57:46.493596 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162460a8-e397-4b7b-aaaf-6ee81d712c51 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:46.494195 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162460a8-e397-4b7b-aaaf-6ee81d712c51 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:46.494424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162460a8-e397-4b7b-aaaf-6ee81d712c51 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:46.496901 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 692/1369] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:57:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:57:46.502939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0852d3-f76a-4986-b246-44d8a6fae155 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:46.503232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0852d3-f76a-4986-b246-44d8a6fae155 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:46.503515 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0852d3-f76a-4986-b246-44d8a6fae155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:46.540606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:46.595494 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-ca0852d3-f76a-4986-b246-44d8a6fae155 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:57:46.596964 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ca0852d3-f76a-4986-b246-44d8a6fae155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:46.723815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 678/1370] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:57:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:57:47.021901 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3aa45-f9f1-4b32-8f96-df5cf33734b1 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:47.022286 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3aa45-f9f1-4b32-8f96-df5cf33734b1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:47.022621 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3aa45-f9f1-4b32-8f96-df5cf33734b1 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:47.024288 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 693/1371] 10.222.0.42 () {54 vars in 964 bytes} [Wed Sep 14 00:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:57:47.030164 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a93ec-aad1-40ae-be40-19ed65bfb2db None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:47.030525 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a93ec-aad1-40ae-be40-19ed65bfb2db None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:47.030883 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a93ec-aad1-40ae-be40-19ed65bfb2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:47.115900 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-973a93ec-aad1-40ae-be40-19ed65bfb2db None None] MFA Rules not processed for user `9373aa297439487691fae325c0245bb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:57:47.117373 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-973a93ec-aad1-40ae-be40-19ed65bfb2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:47.166966 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:47.200971 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 679/1372] 10.222.0.42 () {58 vars in 1061 bytes} [Wed Sep 14 00:57:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 00:57:47.208633 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3aa45-f9f1-4b32-8f96-df5cf33734b1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:57:47.210146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dec3aa45-f9f1-4b32-8f96-df5cf33734b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:47.268033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:47.333486 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3aa45-f9f1-4b32-8f96-df5cf33734b1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:57:47.426749 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98ad4f-5fdb-4568-b8b5-3ac06c1bf12b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:57:47.427479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98ad4f-5fdb-4568-b8b5-3ac06c1bf12b service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:47.427794 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98ad4f-5fdb-4568-b8b5-3ac06c1bf12b service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:47.428130 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98ad4f-5fdb-4568-b8b5-3ac06c1bf12b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:47.429840 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ef98ad4f-5fdb-4568-b8b5-3ac06c1bf12b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:47.553691 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:47.571587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98ad4f-5fdb-4568-b8b5-3ac06c1bf12b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:57:47.573503 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98ad4f-5fdb-4568-b8b5-3ac06c1bf12b service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:57:47.602048 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 694/1373] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:57:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:57:48.738961 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076b7a4-694c-4fb4-a229-4955862ed5d7 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:48.739483 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076b7a4-694c-4fb4-a229-4955862ed5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:48.739483 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076b7a4-694c-4fb4-a229-4955862ed5d7 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:48.740803 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 680/1374] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:57:48.746078 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3de7f6-8b64-4258-876a-76a4744a3f20 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:48.747071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3de7f6-8b64-4258-876a-76a4744a3f20 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:48.747435 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3de7f6-8b64-4258-876a-76a4744a3f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:48.814525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-5f3de7f6-8b64-4258-876a-76a4744a3f20 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:57:48.815787 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5f3de7f6-8b64-4258-876a-76a4744a3f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:48.916251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 695/1375] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:57:49.213309 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076b7a4-694c-4fb4-a229-4955862ed5d7 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:57:49.214873 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1076b7a4-694c-4fb4-a229-4955862ed5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:49.376709 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076b7a4-694c-4fb4-a229-4955862ed5d7 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:57:49.432401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:49.505182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d78b42-3c10-4525-bb4d-7dbda933c89c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:57:49.505807 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d78b42-3c10-4525-bb4d-7dbda933c89c service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:49.506088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d78b42-3c10-4525-bb4d-7dbda933c89c service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:49.506362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d78b42-3c10-4525-bb4d-7dbda933c89c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:49.507990 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-30d78b42-3c10-4525-bb4d-7dbda933c89c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:49.607612 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d78b42-3c10-4525-bb4d-7dbda933c89c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:57:49.609138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d78b42-3c10-4525-bb4d-7dbda933c89c service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:57:49.628990 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 681/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:57:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:57:51.903706 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0eee5-9155-4194-96f7-b06a983553f7 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:51.904242 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0eee5-9155-4194-96f7-b06a983553f7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:51.904242 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0eee5-9155-4194-96f7-b06a983553f7 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:51.905680 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 696/1377] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:57:51.911720 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6346e9-4fc1-4fd4-acd3-2f9cb8dfd8af None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:51.912302 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6346e9-4fc1-4fd4-acd3-2f9cb8dfd8af None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:51.912302 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6346e9-4fc1-4fd4-acd3-2f9cb8dfd8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:51.986854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-2e6346e9-4fc1-4fd4-acd3-2f9cb8dfd8af None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:57:51.988098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2e6346e9-4fc1-4fd4-acd3-2f9cb8dfd8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:51.995892 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:52.093312 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 682/1378] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:57:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:57:52.392140 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b0eee5-9155-4194-96f7-b06a983553f7 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:57:52.393817 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d4b0eee5-9155-4194-96f7-b06a983553f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:52.460928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:52.560599 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b0eee5-9155-4194-96f7-b06a983553f7 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:57:52.682411 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39bed5-c0ee-4994-a995-bacd0fd608e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:57:52.683029 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39bed5-c0ee-4994-a995-bacd0fd608e7 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:52.683313 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39bed5-c0ee-4994-a995-bacd0fd608e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:52.683703 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39bed5-c0ee-4994-a995-bacd0fd608e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:52.685306 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0d39bed5-c0ee-4994-a995-bacd0fd608e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:52.705826 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:52.787620 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39bed5-c0ee-4994-a995-bacd0fd608e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:57:52.789291 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39bed5-c0ee-4994-a995-bacd0fd608e7 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:57:52.809141 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 697/1379] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:57:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:57:55.049184 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692fcc5-5bef-410a-a131-b797988400ee None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:55.049755 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692fcc5-5bef-410a-a131-b797988400ee None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:55.049755 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692fcc5-5bef-410a-a131-b797988400ee None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:55.056361 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57406594-c25f-4299-82cc-75b7dee9ecca None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:55.056361 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57406594-c25f-4299-82cc-75b7dee9ecca None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:55.057093 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57406594-c25f-4299-82cc-75b7dee9ecca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:55.062241 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 683/1380] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:57:55 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:57:55.168818 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-57406594-c25f-4299-82cc-75b7dee9ecca None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:57:55.170047 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-57406594-c25f-4299-82cc-75b7dee9ecca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:55.268680 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:55.275263 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 698/1381] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 00:57:55.581441 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692fcc5-5bef-410a-a131-b797988400ee None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:57:55.583426 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9692fcc5-5bef-410a-a131-b797988400ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:55.738589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:55.750141 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692fcc5-5bef-410a-a131-b797988400ee None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:57:55.897723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2faa3c-d3be-49a1-b06b-a8e2d6589a35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:57:55.898338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2faa3c-d3be-49a1-b06b-a8e2d6589a35 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:55.898681 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2faa3c-d3be-49a1-b06b-a8e2d6589a35 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:55.898964 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2faa3c-d3be-49a1-b06b-a8e2d6589a35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:55.900626 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0b2faa3c-d3be-49a1-b06b-a8e2d6589a35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:56.005339 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2faa3c-d3be-49a1-b06b-a8e2d6589a35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:57:56.006807 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2faa3c-d3be-49a1-b06b-a8e2d6589a35 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:57:56.017331 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:56.026920 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 684/1382] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:57:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:57:58.345205 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3bb22-3fbf-40de-b9fa-31e8d75989d1 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:58.345845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3bb22-3fbf-40de-b9fa-31e8d75989d1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:58.345845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3bb22-3fbf-40de-b9fa-31e8d75989d1 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:58.347103 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 699/1383] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:57:58.352356 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e3852-9efa-426f-a0bf-4e9207e297cd None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:58.352777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e3852-9efa-426f-a0bf-4e9207e297cd None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:58.352887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e3852-9efa-426f-a0bf-4e9207e297cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:58.423987 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-303e3852-9efa-426f-a0bf-4e9207e297cd None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:57:58.425205 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-303e3852-9efa-426f-a0bf-4e9207e297cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:58.544953 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 685/1384] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:57:58.836126 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3bb22-3fbf-40de-b9fa-31e8d75989d1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:57:58.837762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-12b3bb22-3fbf-40de-b9fa-31e8d75989d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:58.998305 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3bb22-3fbf-40de-b9fa-31e8d75989d1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:57:59.069302 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:57:59.119934 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11288c5f-0349-42c3-b8b4-9385f1eb3a55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:57:59.120547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11288c5f-0349-42c3-b8b4-9385f1eb3a55 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:57:59.121018 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11288c5f-0349-42c3-b8b4-9385f1eb3a55 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:57:59.121499 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11288c5f-0349-42c3-b8b4-9385f1eb3a55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:57:59.123718 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-11288c5f-0349-42c3-b8b4-9385f1eb3a55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:57:59.226060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11288c5f-0349-42c3-b8b4-9385f1eb3a55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:57:59.228162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11288c5f-0349-42c3-b8b4-9385f1eb3a55 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:57:59.249370 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 700/1385] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:57:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:58:01.440608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07e5da-361a-49ed-9636-b234ce6a9e71 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:01.441178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07e5da-361a-49ed-9636-b234ce6a9e71 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:01.441178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07e5da-361a-49ed-9636-b234ce6a9e71 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:01.442489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 686/1386] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:58:01.447725 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab82d3b-0458-42b3-b8a3-0cd686d97e13 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:01.448204 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab82d3b-0458-42b3-b8a3-0cd686d97e13 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:01.448302 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab82d3b-0458-42b3-b8a3-0cd686d97e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:01.522183 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-fab82d3b-0458-42b3-b8a3-0cd686d97e13 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:58:01.523378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fab82d3b-0458-42b3-b8a3-0cd686d97e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:01.556590 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:01.621103 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 701/1387] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:58:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:58:01.916914 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07e5da-361a-49ed-9636-b234ce6a9e71 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:58:01.919383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-be07e5da-361a-49ed-9636-b234ce6a9e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:01.997898 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:02.068942 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07e5da-361a-49ed-9636-b234ce6a9e71 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:58:02.181394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fdf8f-edcc-4a0c-96b8-e88ded316030 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:58:02.182001 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fdf8f-edcc-4a0c-96b8-e88ded316030 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:02.182276 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fdf8f-edcc-4a0c-96b8-e88ded316030 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:02.182540 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fdf8f-edcc-4a0c-96b8-e88ded316030 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:02.184563 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d92fdf8f-edcc-4a0c-96b8-e88ded316030 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:02.222681 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:02.274576 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92fdf8f-edcc-4a0c-96b8-e88ded316030 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:58:02.276239 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92fdf8f-edcc-4a0c-96b8-e88ded316030 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:58:02.295534 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 687/1388] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:58:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:58:02.915254 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0e862-f821-48e6-8b5f-cfab601a9cca None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:02.915805 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0e862-f821-48e6-8b5f-cfab601a9cca None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:02.915923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0e862-f821-48e6-8b5f-cfab601a9cca None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:02.917208 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 702/1389] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:58:02.923094 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05443570-4875-4cc2-89ae-5b3801965f16 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:02.923538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05443570-4875-4cc2-89ae-5b3801965f16 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:02.923695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05443570-4875-4cc2-89ae-5b3801965f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:02.991031 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-05443570-4875-4cc2-89ae-5b3801965f16 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:58:03.013849 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-05443570-4875-4cc2-89ae-5b3801965f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:03.103857 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 688/1390] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:58:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:58:03.406211 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01930823-1010-440c-be0d-26e44d85077f None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:03.406502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01930823-1010-440c-be0d-26e44d85077f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:03.406779 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01930823-1010-440c-be0d-26e44d85077f None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:03.408161 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 703/1391] 10.222.0.42 () {54 vars in 966 bytes} [Wed Sep 14 00:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:58:03.413583 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b5b5a-46c2-4075-8bfd-5bed3ae40bd6 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:03.413876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b5b5a-46c2-4075-8bfd-5bed3ae40bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:03.414155 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b5b5a-46c2-4075-8bfd-5bed3ae40bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:03.429898 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:03.495431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-9f3b5b5a-46c2-4075-8bfd-5bed3ae40bd6 None None] MFA Rules not processed for user `89ff357209194276bc079d0c05898747`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:58:03.496679 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9f3b5b5a-46c2-4075-8bfd-5bed3ae40bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:03.574772 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 689/1392] 10.222.0.42 () {58 vars in 1063 bytes} [Wed Sep 14 00:58:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:58:03.584082 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01930823-1010-440c-be0d-26e44d85077f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:58:03.585648 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-01930823-1010-440c-be0d-26e44d85077f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:03.690427 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01930823-1010-440c-be0d-26e44d85077f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:58:03.750471 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:03.767244 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab3e7ee-2092-414a-84aa-f055c3b3cbb3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89ff357209194276bc079d0c05898747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:58:03.767897 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3e7ee-2092-414a-84aa-f055c3b3cbb3 service glance] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:03.768173 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3e7ee-2092-414a-84aa-f055c3b3cbb3 service glance] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:03.768443 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3e7ee-2092-414a-84aa-f055c3b3cbb3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:03.770011 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bab3e7ee-2092-414a-84aa-f055c3b3cbb3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:03.880555 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab3e7ee-2092-414a-84aa-f055c3b3cbb3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:58:03.882078 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab3e7ee-2092-414a-84aa-f055c3b3cbb3 service glance] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:58:03.901809 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 704/1393] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:58:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:58:04.518275 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44ce9f-93df-4933-8acc-ff4c38e3603c None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:04.518869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44ce9f-93df-4933-8acc-ff4c38e3603c None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:04.518869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44ce9f-93df-4933-8acc-ff4c38e3603c None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:04.520297 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 690/1394] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:58:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:58:04.526237 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b72fc-259c-4ade-8f3e-a11a8d7e12ad None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:04.527309 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b72fc-259c-4ade-8f3e-a11a8d7e12ad None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:04.527309 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b72fc-259c-4ade-8f3e-a11a8d7e12ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:04.597694 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7e2b72fc-259c-4ade-8f3e-a11a8d7e12ad None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:58:04.598915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7e2b72fc-259c-4ade-8f3e-a11a8d7e12ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:04.644777 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:04.667583 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40d7b7-959b-41b0-8b73-cc803faac61b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:04.667964 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40d7b7-959b-41b0-8b73-cc803faac61b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:04.668950 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40d7b7-959b-41b0-8b73-cc803faac61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:04.707491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 705/1395] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:58:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:58:04.740149 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-0a40d7b7-959b-41b0-8b73-cc803faac61b None None] MFA Rules not processed for user `ceccc909529042539dd2d5015058d8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:58:04.741372 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0a40d7b7-959b-41b0-8b73-cc803faac61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:04.796597 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:04.828923 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 691/1396] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 00:58:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 00:58:04.876104 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868883d-ccb5-41cc-ab86-611fad5e449c None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:04.876470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868883d-ccb5-41cc-ab86-611fad5e449c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:04.876826 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868883d-ccb5-41cc-ab86-611fad5e449c None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:04.878416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 706/1397] 10.222.0.42 () {54 vars in 943 bytes} [Wed Sep 14 00:58:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:58:04.885021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ebb14-3a95-4912-9c2a-e971f17de3d9 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:04.885384 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ebb14-3a95-4912-9c2a-e971f17de3d9 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:04.885738 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ebb14-3a95-4912-9c2a-e971f17de3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:04.963748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-976ebb14-3a95-4912-9c2a-e971f17de3d9 None None] MFA Rules not processed for user `e52776d9b4894af9b85a24136a8255e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:58:04.965025 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-976ebb14-3a95-4912-9c2a-e971f17de3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:05.002078 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868883d-ccb5-41cc-ab86-611fad5e449c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:58:05.003746 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a868883d-ccb5-41cc-ab86-611fad5e449c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:05.076423 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 692/1398] 10.222.0.42 () {58 vars in 1040 bytes} [Wed Sep 14 00:58:04 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:58:05.087020 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ebb14-3a95-4912-9c2a-e971f17de3d9 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:58:05.099647 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ebb14-3a95-4912-9c2a-e971f17de3d9 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:58:05.127662 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:05.159465 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868883d-ccb5-41cc-ab86-611fad5e449c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:58:05.186649 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:05.219958 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f1dcb-c77a-4638-be63-b9c86d9b7aed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e52776d9b4894af9b85a24136a8255e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:58:05.220637 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f1dcb-c77a-4638-be63-b9c86d9b7aed service swift] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:05.220913 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f1dcb-c77a-4638-be63-b9c86d9b7aed service swift] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:05.221353 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f1dcb-c77a-4638-be63-b9c86d9b7aed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:05.222983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-026f1dcb-c77a-4638-be63-b9c86d9b7aed service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:05.280165 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc27e0-abd4-4124-8d7e-a2e96fe636cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:58:05.280805 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc27e0-abd4-4124-8d7e-a2e96fe636cc service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:05.281088 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc27e0-abd4-4124-8d7e-a2e96fe636cc service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:05.281358 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc27e0-abd4-4124-8d7e-a2e96fe636cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:05.282983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f5cc27e0-abd4-4124-8d7e-a2e96fe636cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:05.323533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f1dcb-c77a-4638-be63-b9c86d9b7aed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:58:05.325564 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f1dcb-c77a-4638-be63-b9c86d9b7aed service swift] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:58:05.334441 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 693/1399] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 00:58:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 14 00:58:05.382900 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc27e0-abd4-4124-8d7e-a2e96fe636cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:58:05.384833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc27e0-abd4-4124-8d7e-a2e96fe636cc service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:58:05.406078 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 707/1400] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:58:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 00:58:07.659183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a00d4-7661-4252-8a07-efb7f2ccae7a None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:07.659746 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a00d4-7661-4252-8a07-efb7f2ccae7a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:07.659746 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a00d4-7661-4252-8a07-efb7f2ccae7a None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:07.661117 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 694/1401] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:58:07.666346 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33973f66-9a17-4ec0-a844-da3325ce364d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:07.666777 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33973f66-9a17-4ec0-a844-da3325ce364d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:07.667446 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33973f66-9a17-4ec0-a844-da3325ce364d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:07.708385 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:07.743689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-33973f66-9a17-4ec0-a844-da3325ce364d None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:58:07.745614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-33973f66-9a17-4ec0-a844-da3325ce364d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:07.864377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 708/1402] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:58:08.170580 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a00d4-7661-4252-8a07-efb7f2ccae7a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:58:08.175364 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ce2a00d4-7661-4252-8a07-efb7f2ccae7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:08.287796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:08.386923 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a00d4-7661-4252-8a07-efb7f2ccae7a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:58:08.514451 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251b8d4-0378-476a-a856-7e77bf984e66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:58:08.515066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251b8d4-0378-476a-a856-7e77bf984e66 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:08.515349 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251b8d4-0378-476a-a856-7e77bf984e66 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:08.515652 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251b8d4-0378-476a-a856-7e77bf984e66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:08.517423 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7251b8d4-0378-476a-a856-7e77bf984e66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:08.538677 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:08.661734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7251b8d4-0378-476a-a856-7e77bf984e66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:58:08.664005 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7251b8d4-0378-476a-a856-7e77bf984e66 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:58:08.684840 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 695/1403] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:58:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:58:10.937609 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b466d8f-eb1d-43e5-9840-a1b9b2121aaa None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:10.939002 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b466d8f-eb1d-43e5-9840-a1b9b2121aaa None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:10.939591 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b466d8f-eb1d-43e5-9840-a1b9b2121aaa None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:10.941284 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 709/1404] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:58:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:58:10.946924 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245966be-67ab-4eb8-8b86-3f584ede0eb9 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:10.947373 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245966be-67ab-4eb8-8b86-3f584ede0eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:10.947469 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245966be-67ab-4eb8-8b86-3f584ede0eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:11.027456 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-245966be-67ab-4eb8-8b86-3f584ede0eb9 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:58:11.028744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-245966be-67ab-4eb8-8b86-3f584ede0eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:11.131070 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:11.139035 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 696/1405] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:58:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:58:11.430585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b466d8f-eb1d-43e5-9840-a1b9b2121aaa None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:58:11.432275 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1b466d8f-eb1d-43e5-9840-a1b9b2121aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:11.557040 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:11.607258 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b466d8f-eb1d-43e5-9840-a1b9b2121aaa None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:58:11.730087 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83937d-1867-4288-81cf-863262d75e02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:58:11.730618 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83937d-1867-4288-81cf-863262d75e02 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:11.730896 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83937d-1867-4288-81cf-863262d75e02 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:11.731173 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83937d-1867-4288-81cf-863262d75e02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:11.732808 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6b83937d-1867-4288-81cf-863262d75e02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:11.814360 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:11.835462 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83937d-1867-4288-81cf-863262d75e02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:58:11.837232 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b83937d-1867-4288-81cf-863262d75e02 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:58:11.859954 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 710/1406] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:58:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:58:14.092582 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401f972-db23-4102-9f65-f9a77f67946d None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:14.093177 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401f972-db23-4102-9f65-f9a77f67946d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:14.093177 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401f972-db23-4102-9f65-f9a77f67946d None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:14.094475 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 697/1407] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:58:14.099884 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b1df8-2328-4a0c-bd1a-fb075ed448df None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:14.100309 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b1df8-2328-4a0c-bd1a-fb075ed448df None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:14.100520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b1df8-2328-4a0c-bd1a-fb075ed448df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:14.176675 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ae0b1df8-2328-4a0c-bd1a-fb075ed448df None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:58:14.177920 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ae0b1df8-2328-4a0c-bd1a-fb075ed448df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:14.290903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 711/1408] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:58:14.588511 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401f972-db23-4102-9f65-f9a77f67946d None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:58:14.590295 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-3401f972-db23-4102-9f65-f9a77f67946d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:14.756225 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401f972-db23-4102-9f65-f9a77f67946d None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:58:14.832514 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:14.895281 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b960313-c612-4937-ad8c-bf250c384c77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:58:14.896038 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b960313-c612-4937-ad8c-bf250c384c77 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:14.896388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b960313-c612-4937-ad8c-bf250c384c77 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:14.896731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b960313-c612-4937-ad8c-bf250c384c77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:14.898420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0b960313-c612-4937-ad8c-bf250c384c77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:15.002494 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b960313-c612-4937-ad8c-bf250c384c77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:58:15.004140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b960313-c612-4937-ad8c-bf250c384c77 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:58:15.025794 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 698/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:58:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:58:24.289079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=79 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:25.023211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=64 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:28.547321 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c849b-5c4f-4d2b-8b66-bb9d249d47ed None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:28.547905 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c849b-5c4f-4d2b-8b66-bb9d249d47ed None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:28.547905 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c849b-5c4f-4d2b-8b66-bb9d249d47ed None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:28.549228 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 712/1410] 10.222.0.42 () {54 vars in 918 bytes} [Wed Sep 14 00:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:58:28.554323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1611b2d-eb36-47ff-a849-952dbfe83292 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:28.554746 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1611b2d-eb36-47ff-a849-952dbfe83292 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:28.554849 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1611b2d-eb36-47ff-a849-952dbfe83292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:28.633616 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-b1611b2d-eb36-47ff-a849-952dbfe83292 None None] MFA Rules not processed for user `18798838de284c84b5b838b3530a2c43`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:58:28.635082 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b1611b2d-eb36-47ff-a849-952dbfe83292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:28.741340 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 699/1411] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Sep 14 00:58:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:58:28.756408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61bb52-8c9f-40ed-a35b-138f22b9afa4 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:28.756696 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61bb52-8c9f-40ed-a35b-138f22b9afa4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:28.756971 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61bb52-8c9f-40ed-a35b-138f22b9afa4 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:28.758290 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 713/1412] 10.222.0.42 () {54 vars in 918 bytes} [Wed Sep 14 00:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:58:28.763180 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772e01b-745e-485e-8493-fa4b93f071ab None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:28.763465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772e01b-745e-485e-8493-fa4b93f071ab None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:28.763763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772e01b-745e-485e-8493-fa4b93f071ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:28.835634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:28.838924 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-9772e01b-745e-485e-8493-fa4b93f071ab None None] MFA Rules not processed for user `5e0fe2e84f6748c992c9beedf2f5d49b`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:58:28.841255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9772e01b-745e-485e-8493-fa4b93f071ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:28.955889 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 700/1413] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Sep 14 00:58:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:58:31.552227 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813d704-479e-4001-8d02-14e3f1808b4f None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:31.552771 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813d704-479e-4001-8d02-14e3f1808b4f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:31.552873 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813d704-479e-4001-8d02-14e3f1808b4f None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:31.554138 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 714/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 00:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:58:31.559986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4ed91-5d76-4f7b-a22c-f1f8c3baef52 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:31.560424 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4ed91-5d76-4f7b-a22c-f1f8c3baef52 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:31.560554 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4ed91-5d76-4f7b-a22c-f1f8c3baef52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:31.626601 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-c6c4ed91-5d76-4f7b-a22c-f1f8c3baef52 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:58:31.627836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c6c4ed91-5d76-4f7b-a22c-f1f8c3baef52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:31.683687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:31.725065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 701/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 00:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:58:32.019070 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813d704-479e-4001-8d02-14e3f1808b4f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:58:32.020666 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b813d704-479e-4001-8d02-14e3f1808b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:32.175089 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813d704-479e-4001-8d02-14e3f1808b4f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:58:32.251217 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:32.291930 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9290d5-e9a4-4ebe-89c6-3d73c2a5fd05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:58:32.292495 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9290d5-e9a4-4ebe-89c6-3d73c2a5fd05 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:32.292779 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9290d5-e9a4-4ebe-89c6-3d73c2a5fd05 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:32.293055 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9290d5-e9a4-4ebe-89c6-3d73c2a5fd05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:32.294654 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3e9290d5-e9a4-4ebe-89c6-3d73c2a5fd05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:32.383608 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9290d5-e9a4-4ebe-89c6-3d73c2a5fd05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:58:32.385519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9290d5-e9a4-4ebe-89c6-3d73c2a5fd05 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:58:32.403922 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 715/1416] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:58:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:58:33.553557 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8a012-1343-4153-bb58-02285d63cbda None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:33.554112 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8a012-1343-4153-bb58-02285d63cbda None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:33.554112 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8a012-1343-4153-bb58-02285d63cbda None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:33.555446 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 702/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 00:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 00:58:33.561952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da508e-f276-49fe-881d-fb892b6ce70e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:33.562400 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da508e-f276-49fe-881d-fb892b6ce70e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:33.562524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da508e-f276-49fe-881d-fb892b6ce70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:33.637941 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-51da508e-f276-49fe-881d-fb892b6ce70e None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:58:33.639132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-51da508e-f276-49fe-881d-fb892b6ce70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:33.666117 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:33.737224 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 716/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 00:58:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 00:58:33.897097 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8a012-1343-4153-bb58-02285d63cbda None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:58:33.898613 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a1f8a012-1343-4153-bb58-02285d63cbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:34.045355 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8a012-1343-4153-bb58-02285d63cbda None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:58:34.085998 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:34.159025 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9890e0b-06cd-43ff-b3d7-faf84d7fccf5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:58:34.159659 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9890e0b-06cd-43ff-b3d7-faf84d7fccf5 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:34.159939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9890e0b-06cd-43ff-b3d7-faf84d7fccf5 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:34.160210 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9890e0b-06cd-43ff-b3d7-faf84d7fccf5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:34.162118 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e9890e0b-06cd-43ff-b3d7-faf84d7fccf5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:34.250906 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9890e0b-06cd-43ff-b3d7-faf84d7fccf5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:58:34.252551 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9890e0b-06cd-43ff-b3d7-faf84d7fccf5 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:58:34.272141 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 703/1419] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:58:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:58:35.400639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b3cdb-cc98-46dd-892f-327fe3a95e7b None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:35.401185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b3cdb-cc98-46dd-892f-327fe3a95e7b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:35.401185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b3cdb-cc98-46dd-892f-327fe3a95e7b None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:35.403435 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 717/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 00:58:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:58:35.409728 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad21cee-1705-4887-8863-75f959a4f09a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:35.410151 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad21cee-1705-4887-8863-75f959a4f09a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:35.410308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad21cee-1705-4887-8863-75f959a4f09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:35.464802 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:35.487917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-aad21cee-1705-4887-8863-75f959a4f09a None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:58:35.489135 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-aad21cee-1705-4887-8863-75f959a4f09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:35.586663 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 704/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 00:58:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:58:35.739022 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b3cdb-cc98-46dd-892f-327fe3a95e7b None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:58:35.740678 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-086b3cdb-cc98-46dd-892f-327fe3a95e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:35.890189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b3cdb-cc98-46dd-892f-327fe3a95e7b None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:58:35.902146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:36.003385 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9a999-d08f-4b8a-b212-1bfc9a5f0994 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:58:36.003937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9a999-d08f-4b8a-b212-1bfc9a5f0994 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:36.004214 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9a999-d08f-4b8a-b212-1bfc9a5f0994 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:36.004484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9a999-d08f-4b8a-b212-1bfc9a5f0994 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:36.006063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-15d9a999-d08f-4b8a-b212-1bfc9a5f0994 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:36.096513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d9a999-d08f-4b8a-b212-1bfc9a5f0994 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:58:36.098111 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d9a999-d08f-4b8a-b212-1bfc9a5f0994 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:58:36.116558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 718/1422] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:58:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:58:37.227174 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03663472-1603-4c0d-8e4b-9633e39121be None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:37.227806 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03663472-1603-4c0d-8e4b-9633e39121be None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:37.227806 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03663472-1603-4c0d-8e4b-9633e39121be None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:37.229083 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 705/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 00:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:58:37.235919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589998d-f09f-43c2-a24d-b86a04396f8a None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:37.236420 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589998d-f09f-43c2-a24d-b86a04396f8a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:37.236529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589998d-f09f-43c2-a24d-b86a04396f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:37.256691 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:37.313728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-3589998d-f09f-43c2-a24d-b86a04396f8a None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:58:37.314937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3589998d-f09f-43c2-a24d-b86a04396f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:37.411105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 719/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 00:58:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:58:37.562607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03663472-1603-4c0d-8e4b-9633e39121be None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:58:37.564188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-03663472-1603-4c0d-8e4b-9633e39121be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:37.674360 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:37.710550 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03663472-1603-4c0d-8e4b-9633e39121be None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:58:37.827049 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb86d99-1beb-4166-afff-4f5edd7a4100 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:58:37.827591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb86d99-1beb-4166-afff-4f5edd7a4100 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:37.827874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb86d99-1beb-4166-afff-4f5edd7a4100 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:37.828143 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb86d99-1beb-4166-afff-4f5edd7a4100 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:37.829718 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0cb86d99-1beb-4166-afff-4f5edd7a4100 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:37.907452 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:37.920992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb86d99-1beb-4166-afff-4f5edd7a4100 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:58:37.922591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb86d99-1beb-4166-afff-4f5edd7a4100 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:58:37.940443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 706/1425] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:58:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:58:39.076089 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10529d45-64b3-4cc2-a09b-c5088babd27d None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:39.076756 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10529d45-64b3-4cc2-a09b-c5088babd27d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:39.076756 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10529d45-64b3-4cc2-a09b-c5088babd27d None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:39.078157 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 720/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 00:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:58:39.084393 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2a958-647f-40c1-8993-b12a803b8edf None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:39.084835 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2a958-647f-40c1-8993-b12a803b8edf None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:39.084940 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2a958-647f-40c1-8993-b12a803b8edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:39.162057 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-cbe2a958-647f-40c1-8993-b12a803b8edf None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:58:39.163260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-cbe2a958-647f-40c1-8993-b12a803b8edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:39.260231 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 707/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 00:58:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:58:39.414434 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10529d45-64b3-4cc2-a09b-c5088babd27d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:58:39.415999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-10529d45-64b3-4cc2-a09b-c5088babd27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:39.501351 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:39.564580 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10529d45-64b3-4cc2-a09b-c5088babd27d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:58:39.675892 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7ef72-1e52-4f9f-b979-9f0779fc89c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:58:39.676434 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7ef72-1e52-4f9f-b979-9f0779fc89c3 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:39.676715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7ef72-1e52-4f9f-b979-9f0779fc89c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:39.676988 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7ef72-1e52-4f9f-b979-9f0779fc89c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:39.678565 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6cb7ef72-1e52-4f9f-b979-9f0779fc89c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:39.727468 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:39.770500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb7ef72-1e52-4f9f-b979-9f0779fc89c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:58:39.772113 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb7ef72-1e52-4f9f-b979-9f0779fc89c3 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:58:39.789798 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 721/1428] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:58:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:58:40.842488 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3c612-0586-4d31-bfa8-857110f3626a None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:40.843034 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3c612-0586-4d31-bfa8-857110f3626a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:40.843034 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3c612-0586-4d31-bfa8-857110f3626a None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:40.844407 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 708/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 00:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:58:40.850557 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec5ecb-9e50-4e5f-b1e0-a0a1b0ec1e81 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:40.850973 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec5ecb-9e50-4e5f-b1e0-a0a1b0ec1e81 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:40.851110 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec5ecb-9e50-4e5f-b1e0-a0a1b0ec1e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:40.933561 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-63ec5ecb-9e50-4e5f-b1e0-a0a1b0ec1e81 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:58:40.934795 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-63ec5ecb-9e50-4e5f-b1e0-a0a1b0ec1e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:41.032816 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 722/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 00:58:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:58:41.187749 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd3c612-0586-4d31-bfa8-857110f3626a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:58:41.189260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-3bd3c612-0586-4d31-bfa8-857110f3626a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:41.237994 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:41.335024 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd3c612-0586-4d31-bfa8-857110f3626a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:58:41.446583 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e096a-271f-443f-b25d-b4c3e8a4275e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:58:41.447127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e096a-271f-443f-b25d-b4c3e8a4275e service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:41.447411 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e096a-271f-443f-b25d-b4c3e8a4275e service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:41.447719 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e096a-271f-443f-b25d-b4c3e8a4275e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:41.449382 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-af0e096a-271f-443f-b25d-b4c3e8a4275e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:41.456245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:41.539387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e096a-271f-443f-b25d-b4c3e8a4275e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:58:41.541024 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e096a-271f-443f-b25d-b4c3e8a4275e service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:58:41.559813 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 709/1431] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:58:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:58:42.619417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40448f7-6e20-431a-a317-e0ac47983cfe None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:42.619858 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40448f7-6e20-431a-a317-e0ac47983cfe None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:42.620027 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40448f7-6e20-431a-a317-e0ac47983cfe None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:42.621414 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 723/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 00:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:58:42.627963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfdaf1-e14e-4d13-bf9a-f91205776e0d None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:42.628432 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfdaf1-e14e-4d13-bf9a-f91205776e0d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:42.628527 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfdaf1-e14e-4d13-bf9a-f91205776e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:42.705795 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-15bfdaf1-e14e-4d13-bf9a-f91205776e0d None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:58:42.707022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-15bfdaf1-e14e-4d13-bf9a-f91205776e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:42.774376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:42.803939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 710/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 00:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:58:42.955490 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40448f7-6e20-431a-a317-e0ac47983cfe None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:58:42.957060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f40448f7-6e20-431a-a317-e0ac47983cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:42.975006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:43.102816 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40448f7-6e20-431a-a317-e0ac47983cfe None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:58:43.189556 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:43.215598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe3007-2956-46a3-b3c2-0f8ffc586c7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:58:43.216210 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe3007-2956-46a3-b3c2-0f8ffc586c7d service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:43.216491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe3007-2956-46a3-b3c2-0f8ffc586c7d service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:43.216765 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe3007-2956-46a3-b3c2-0f8ffc586c7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:43.218352 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6ffe3007-2956-46a3-b3c2-0f8ffc586c7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:43.307353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe3007-2956-46a3-b3c2-0f8ffc586c7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:58:43.309633 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe3007-2956-46a3-b3c2-0f8ffc586c7d service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:58:43.328460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 724/1434] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:58:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:58:44.379800 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1babc3b-4314-4b79-9286-3ac7130d8874 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:44.380385 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1babc3b-4314-4b79-9286-3ac7130d8874 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:44.380385 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1babc3b-4314-4b79-9286-3ac7130d8874 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:44.381678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 711/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 00:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:58:44.388742 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41d83f-10e7-41d5-ac10-2be29651084d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:44.389246 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41d83f-10e7-41d5-ac10-2be29651084d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:44.389345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41d83f-10e7-41d5-ac10-2be29651084d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:44.464735 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-fc41d83f-10e7-41d5-ac10-2be29651084d None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:58:44.465908 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fc41d83f-10e7-41d5-ac10-2be29651084d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:44.509027 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:44.563709 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 725/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 00:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:58:44.717417 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1babc3b-4314-4b79-9286-3ac7130d8874 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:58:44.718931 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e1babc3b-4314-4b79-9286-3ac7130d8874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:44.863213 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1babc3b-4314-4b79-9286-3ac7130d8874 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:58:44.921563 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:44.978694 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c888edfb-e926-439d-afa0-d9f9539e4854 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:58:44.979301 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c888edfb-e926-439d-afa0-d9f9539e4854 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:44.980307 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c888edfb-e926-439d-afa0-d9f9539e4854 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:44.980584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c888edfb-e926-439d-afa0-d9f9539e4854 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:44.982163 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c888edfb-e926-439d-afa0-d9f9539e4854 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:45.070260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c888edfb-e926-439d-afa0-d9f9539e4854 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:58:45.071896 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c888edfb-e926-439d-afa0-d9f9539e4854 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:58:45.089782 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 712/1437] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:58:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:58:46.146965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982d28b-16fd-4f33-ad1f-ef3cceabbc12 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:46.147537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982d28b-16fd-4f33-ad1f-ef3cceabbc12 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:46.147537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982d28b-16fd-4f33-ad1f-ef3cceabbc12 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:46.148940 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 726/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 00:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:58:46.156266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c70e4e-f26e-415c-a072-2326a4aa11bd None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:46.156741 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c70e4e-f26e-415c-a072-2326a4aa11bd None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:46.156849 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c70e4e-f26e-415c-a072-2326a4aa11bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:46.234130 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-28c70e4e-f26e-415c-a072-2326a4aa11bd None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:58:46.235627 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-28c70e4e-f26e-415c-a072-2326a4aa11bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:46.237638 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:46.334635 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 713/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 00:58:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:58:46.486204 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982d28b-16fd-4f33-ad1f-ef3cceabbc12 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:58:46.487773 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4982d28b-16fd-4f33-ad1f-ef3cceabbc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:46.631178 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982d28b-16fd-4f33-ad1f-ef3cceabbc12 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:58:46.659361 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:46.744290 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c59b5-e390-46e8-b2b9-e02e82ad9a33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:58:46.744832 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c59b5-e390-46e8-b2b9-e02e82ad9a33 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:46.745113 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c59b5-e390-46e8-b2b9-e02e82ad9a33 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:46.745446 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c59b5-e390-46e8-b2b9-e02e82ad9a33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:46.747141 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-887c59b5-e390-46e8-b2b9-e02e82ad9a33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:46.835093 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c59b5-e390-46e8-b2b9-e02e82ad9a33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:58:46.836725 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c59b5-e390-46e8-b2b9-e02e82ad9a33 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:58:46.854428 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 727/1440] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:58:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:58:48.950116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b077139-85a0-43ff-86e2-169ea9187f22 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:48.950674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b077139-85a0-43ff-86e2-169ea9187f22 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:48.950674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b077139-85a0-43ff-86e2-169ea9187f22 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:48.952103 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 714/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 00:58:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:58:48.957449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b229e-bc2c-4453-8481-c8d06575c517 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:48.957861 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b229e-bc2c-4453-8481-c8d06575c517 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:48.958000 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b229e-bc2c-4453-8481-c8d06575c517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:48.998856 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:49.025391 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-dc0b229e-bc2c-4453-8481-c8d06575c517 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:58:49.027338 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dc0b229e-bc2c-4453-8481-c8d06575c517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:49.125074 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 728/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 00:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:58:49.413553 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b077139-85a0-43ff-86e2-169ea9187f22 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:58:49.415076 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0b077139-85a0-43ff-86e2-169ea9187f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:49.518910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b077139-85a0-43ff-86e2-169ea9187f22 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:58:49.550675 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:49.592618 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd155fd-00eb-4953-a7dd-0fc9b920a936 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:58:49.593226 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd155fd-00eb-4953-a7dd-0fc9b920a936 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:49.593504 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd155fd-00eb-4953-a7dd-0fc9b920a936 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:49.593775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd155fd-00eb-4953-a7dd-0fc9b920a936 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:49.595369 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9dd155fd-00eb-4953-a7dd-0fc9b920a936 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:49.704983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd155fd-00eb-4953-a7dd-0fc9b920a936 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:58:49.706428 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd155fd-00eb-4953-a7dd-0fc9b920a936 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:58:49.724631 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 715/1443] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:58:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:58:52.036095 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc55399-db44-40f9-bb2d-21f9d05f4f2c None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:52.036631 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc55399-db44-40f9-bb2d-21f9d05f4f2c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:52.036631 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc55399-db44-40f9-bb2d-21f9d05f4f2c None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:52.038059 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 729/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 00:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:58:52.043975 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038303da-a602-4eb3-a8f4-02cb3746b598 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:52.044424 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038303da-a602-4eb3-a8f4-02cb3746b598 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:52.044523 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038303da-a602-4eb3-a8f4-02cb3746b598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:52.112831 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-038303da-a602-4eb3-a8f4-02cb3746b598 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:58:52.113947 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-038303da-a602-4eb3-a8f4-02cb3746b598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:52.127652 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:52.212807 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 716/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 00:58:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:58:52.497352 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc55399-db44-40f9-bb2d-21f9d05f4f2c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:58:52.498941 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7cc55399-db44-40f9-bb2d-21f9d05f4f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:52.600983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc55399-db44-40f9-bb2d-21f9d05f4f2c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:58:52.613607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:52.675701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1eb2a-6a4d-4647-a435-77269cca5094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:58:52.676315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1eb2a-6a4d-4647-a435-77269cca5094 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:52.676598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1eb2a-6a4d-4647-a435-77269cca5094 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:52.676872 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1eb2a-6a4d-4647-a435-77269cca5094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:52.678456 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6ea1eb2a-6a4d-4647-a435-77269cca5094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:52.786389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1eb2a-6a4d-4647-a435-77269cca5094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:58:52.787772 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1eb2a-6a4d-4647-a435-77269cca5094 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:58:52.805918 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 730/1446] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:58:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:58:55.293687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00dcadb-4b05-41be-8b73-c9f8d4eba428 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:55.294268 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00dcadb-4b05-41be-8b73-c9f8d4eba428 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:55.294268 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00dcadb-4b05-41be-8b73-c9f8d4eba428 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:55.295656 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 717/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 00:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:58:55.300987 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654028c-546f-4a9a-a4cd-c307e2fb6803 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:55.301412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654028c-546f-4a9a-a4cd-c307e2fb6803 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:55.301566 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654028c-546f-4a9a-a4cd-c307e2fb6803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:55.350197 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:55.368225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4654028c-546f-4a9a-a4cd-c307e2fb6803 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:58:55.369417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4654028c-546f-4a9a-a4cd-c307e2fb6803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:55.466416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 731/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 00:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:58:55.754007 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00dcadb-4b05-41be-8b73-c9f8d4eba428 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:58:55.755619 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a00dcadb-4b05-41be-8b73-c9f8d4eba428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:55.858845 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00dcadb-4b05-41be-8b73-c9f8d4eba428 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:58:55.898687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:55.932771 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ecb3b-bf7e-4f23-9610-54ff387e5058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:58:55.933409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ecb3b-bf7e-4f23-9610-54ff387e5058 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:55.933689 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ecb3b-bf7e-4f23-9610-54ff387e5058 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:55.933961 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ecb3b-bf7e-4f23-9610-54ff387e5058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:55.935581 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-555ecb3b-bf7e-4f23-9610-54ff387e5058 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:56.044985 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ecb3b-bf7e-4f23-9610-54ff387e5058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:58:56.046379 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ecb3b-bf7e-4f23-9610-54ff387e5058 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:58:56.064561 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 718/1449] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:58:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:58:58.337803 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae3c0a-7583-4570-bf0e-2672e2b0d6bd None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:58.338355 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae3c0a-7583-4570-bf0e-2672e2b0d6bd None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:58.338355 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae3c0a-7583-4570-bf0e-2672e2b0d6bd None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:58.340289 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 732/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 00:58:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:58:58.345615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017f93c-7c61-4e2f-ad1f-7348aba1dbe9 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:58.346039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017f93c-7c61-4e2f-ad1f-7348aba1dbe9 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:58.346156 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017f93c-7c61-4e2f-ad1f-7348aba1dbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:58.412296 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a017f93c-7c61-4e2f-ad1f-7348aba1dbe9 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:58:58.413483 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a017f93c-7c61-4e2f-ad1f-7348aba1dbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:58.434923 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:58.512665 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 719/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 00:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:58:58.801890 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae3c0a-7583-4570-bf0e-2672e2b0d6bd None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:58:58.803441 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-12ae3c0a-7583-4570-bf0e-2672e2b0d6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:58.907982 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae3c0a-7583-4570-bf0e-2672e2b0d6bd None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:58:58.928196 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:58:58.981713 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9821ae-8f59-4812-a2a8-22ce37c2282d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:58:58.982318 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9821ae-8f59-4812-a2a8-22ce37c2282d service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:58:58.982594 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9821ae-8f59-4812-a2a8-22ce37c2282d service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:58:58.982859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9821ae-8f59-4812-a2a8-22ce37c2282d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:58:58.984465 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cc9821ae-8f59-4812-a2a8-22ce37c2282d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:58:59.091516 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9821ae-8f59-4812-a2a8-22ce37c2282d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:58:59.092977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9821ae-8f59-4812-a2a8-22ce37c2282d service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:58:59.111219 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 733/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:58:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:59:01.297343 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f17a1-4d5e-4438-8897-f05797229c13 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:01.297885 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f17a1-4d5e-4438-8897-f05797229c13 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:01.297885 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f17a1-4d5e-4438-8897-f05797229c13 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:01.299219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 720/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 00:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:59:01.304602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107e71d-f6fb-47c2-bea0-edc949fb58b5 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:01.305013 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107e71d-f6fb-47c2-bea0-edc949fb58b5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:01.305148 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107e71d-f6fb-47c2-bea0-edc949fb58b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:01.367992 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:01.370844 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f107e71d-f6fb-47c2-bea0-edc949fb58b5 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:59:01.372861 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f107e71d-f6fb-47c2-bea0-edc949fb58b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:01.468184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 734/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 00:59:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:59:01.751280 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f17a1-4d5e-4438-8897-f05797229c13 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:59:01.752848 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ee0f17a1-4d5e-4438-8897-f05797229c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:01.855162 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f17a1-4d5e-4438-8897-f05797229c13 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:59:01.904874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:01.929793 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832d3264-71a7-4f6b-a818-314df0cf454e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:59:01.930440 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d3264-71a7-4f6b-a818-314df0cf454e service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:01.930720 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d3264-71a7-4f6b-a818-314df0cf454e service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:01.930996 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d3264-71a7-4f6b-a818-314df0cf454e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:01.932634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-832d3264-71a7-4f6b-a818-314df0cf454e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:02.042760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832d3264-71a7-4f6b-a818-314df0cf454e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:59:02.044277 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832d3264-71a7-4f6b-a818-314df0cf454e service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:59:02.062231 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 721/1455] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:59:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:59:04.299465 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ec821-5b3a-4e1c-9c68-c6b06d004912 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:04.300041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ec821-5b3a-4e1c-9c68-c6b06d004912 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:04.300136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ec821-5b3a-4e1c-9c68-c6b06d004912 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:04.301420 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 735/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 00:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:59:04.306757 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006832ef-34a8-489f-8616-f71229a5f070 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:04.307209 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006832ef-34a8-489f-8616-f71229a5f070 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:04.307307 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006832ef-34a8-489f-8616-f71229a5f070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:04.375054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-006832ef-34a8-489f-8616-f71229a5f070 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:59:04.376274 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-006832ef-34a8-489f-8616-f71229a5f070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:04.405168 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:04.473702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 722/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 00:59:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:59:04.755069 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ec821-5b3a-4e1c-9c68-c6b06d004912 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:59:04.756699 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ca3ec821-5b3a-4e1c-9c68-c6b06d004912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:04.861032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ec821-5b3a-4e1c-9c68-c6b06d004912 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:59:04.890334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:04.937831 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f05c0f5-3644-4534-bb0a-1028d0edf431 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:59:04.938448 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05c0f5-3644-4534-bb0a-1028d0edf431 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:04.938734 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05c0f5-3644-4534-bb0a-1028d0edf431 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:04.939009 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05c0f5-3644-4534-bb0a-1028d0edf431 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:04.940645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3f05c0f5-3644-4534-bb0a-1028d0edf431 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:05.049287 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f05c0f5-3644-4534-bb0a-1028d0edf431 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:59:05.050730 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f05c0f5-3644-4534-bb0a-1028d0edf431 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:59:05.068918 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 736/1458] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:59:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:59:07.292008 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85827430-a7ec-4ab2-884a-c2fb9d5c36d6 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:07.292749 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85827430-a7ec-4ab2-884a-c2fb9d5c36d6 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:07.293030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85827430-a7ec-4ab2-884a-c2fb9d5c36d6 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:07.294388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 723/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 00:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:59:07.299700 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1203f-d5f2-4546-bb53-9352652979fc None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:07.300084 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1203f-d5f2-4546-bb53-9352652979fc None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:07.300283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1203f-d5f2-4546-bb53-9352652979fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:07.365646 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0de1203f-d5f2-4546-bb53-9352652979fc None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:59:07.366849 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0de1203f-d5f2-4546-bb53-9352652979fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:07.375520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:07.463903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 737/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 00:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:59:07.753536 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85827430-a7ec-4ab2-884a-c2fb9d5c36d6 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:59:07.755072 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-85827430-a7ec-4ab2-884a-c2fb9d5c36d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:07.858618 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85827430-a7ec-4ab2-884a-c2fb9d5c36d6 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:59:07.913530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:07.931959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0a378-c43c-47d8-8244-c787c1927681 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:59:07.932565 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0a378-c43c-47d8-8244-c787c1927681 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:07.933438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0a378-c43c-47d8-8244-c787c1927681 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:07.933710 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0a378-c43c-47d8-8244-c787c1927681 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:07.935295 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-eda0a378-c43c-47d8-8244-c787c1927681 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:08.043882 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0a378-c43c-47d8-8244-c787c1927681 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:59:08.045245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0a378-c43c-47d8-8244-c787c1927681 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:59:08.067241 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 724/1461] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:59:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:59:10.444741 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b99f8-42d7-4cd1-8b90-7394e0ad2bb5 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:10.445332 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b99f8-42d7-4cd1-8b90-7394e0ad2bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:10.445332 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b99f8-42d7-4cd1-8b90-7394e0ad2bb5 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:10.446618 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 738/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 00:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:59:10.452806 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277fbd7-d9d7-485c-a373-0a33e2b3bc50 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:10.453222 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277fbd7-d9d7-485c-a373-0a33e2b3bc50 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:10.453353 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277fbd7-d9d7-485c-a373-0a33e2b3bc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:10.519843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-1277fbd7-d9d7-485c-a373-0a33e2b3bc50 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:59:10.521081 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1277fbd7-d9d7-485c-a373-0a33e2b3bc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:10.559363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:10.617021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 725/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 00:59:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:59:10.900139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b99f8-42d7-4cd1-8b90-7394e0ad2bb5 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:59:10.901684 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-618b99f8-42d7-4cd1-8b90-7394e0ad2bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:11.010769 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b99f8-42d7-4cd1-8b90-7394e0ad2bb5 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:59:11.049035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:11.086857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd802b3f-23f1-44f9-8600-eb0cd16d3274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:59:11.087467 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd802b3f-23f1-44f9-8600-eb0cd16d3274 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:11.087788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd802b3f-23f1-44f9-8600-eb0cd16d3274 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:11.088063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd802b3f-23f1-44f9-8600-eb0cd16d3274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:11.089827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dd802b3f-23f1-44f9-8600-eb0cd16d3274 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:11.199625 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd802b3f-23f1-44f9-8600-eb0cd16d3274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:59:11.201067 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd802b3f-23f1-44f9-8600-eb0cd16d3274 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:59:11.219376 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 739/1464] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:59:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:59:13.453205 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70dbc16-68ab-4686-bfbd-0d657683d450 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:13.453769 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70dbc16-68ab-4686-bfbd-0d657683d450 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:13.453769 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70dbc16-68ab-4686-bfbd-0d657683d450 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:13.455083 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 726/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 00:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:59:13.460543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd799f11-f0b2-47ed-8134-1b1da14c9c36 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:13.460963 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd799f11-f0b2-47ed-8134-1b1da14c9c36 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:13.461101 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd799f11-f0b2-47ed-8134-1b1da14c9c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:13.528196 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-bd799f11-f0b2-47ed-8134-1b1da14c9c36 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:59:13.529396 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bd799f11-f0b2-47ed-8134-1b1da14c9c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:13.547812 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:13.626647 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 740/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 00:59:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:59:13.915866 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70dbc16-68ab-4686-bfbd-0d657683d450 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:59:13.917979 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d70dbc16-68ab-4686-bfbd-0d657683d450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:14.021417 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70dbc16-68ab-4686-bfbd-0d657683d450 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:59:14.087487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:14.095099 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9847c468-4178-42e7-8778-53072326d9ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:59:14.095749 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847c468-4178-42e7-8778-53072326d9ce service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:14.096030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847c468-4178-42e7-8778-53072326d9ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:14.096302 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847c468-4178-42e7-8778-53072326d9ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:14.097922 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9847c468-4178-42e7-8778-53072326d9ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:14.207680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9847c468-4178-42e7-8778-53072326d9ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:59:14.208978 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9847c468-4178-42e7-8778-53072326d9ce service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:59:14.226723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 727/1467] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:59:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:59:16.407491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bde550-9ff9-4d2f-84c6-97bc356c6ce9 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:16.408102 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bde550-9ff9-4d2f-84c6-97bc356c6ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:16.408102 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bde550-9ff9-4d2f-84c6-97bc356c6ce9 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:16.409465 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 741/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 00:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:59:16.414660 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb09d3-a4e4-4b91-a276-017cf25c4815 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:16.415058 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb09d3-a4e4-4b91-a276-017cf25c4815 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:16.415218 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb09d3-a4e4-4b91-a276-017cf25c4815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:16.481708 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e0cb09d3-a4e4-4b91-a276-017cf25c4815 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:59:16.482923 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e0cb09d3-a4e4-4b91-a276-017cf25c4815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:16.532215 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:16.580293 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 728/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 00:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:59:16.864244 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bde550-9ff9-4d2f-84c6-97bc356c6ce9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:59:16.865787 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-96bde550-9ff9-4d2f-84c6-97bc356c6ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:16.970118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bde550-9ff9-4d2f-84c6-97bc356c6ce9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:59:17.018459 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:17.045385 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91da4c2-48f6-4544-9e7e-8ac9a669b9be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:59:17.045999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91da4c2-48f6-4544-9e7e-8ac9a669b9be service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:17.046279 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91da4c2-48f6-4544-9e7e-8ac9a669b9be service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:17.046551 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91da4c2-48f6-4544-9e7e-8ac9a669b9be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:17.048169 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d91da4c2-48f6-4544-9e7e-8ac9a669b9be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:17.158602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91da4c2-48f6-4544-9e7e-8ac9a669b9be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:59:17.160024 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91da4c2-48f6-4544-9e7e-8ac9a669b9be service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:59:17.178391 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 742/1470] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:59:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:59:26.578740 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=22 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:27.176164 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=66 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:37.295935 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f2d02-2a3c-47f4-9054-2acbd459033e None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:37.296509 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f2d02-2a3c-47f4-9054-2acbd459033e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:37.296509 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f2d02-2a3c-47f4-9054-2acbd459033e None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:37.297821 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 729/1471] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:59:37.303109 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875f85f-0a9f-4555-b7c4-0ae0245bfdbd None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:37.303591 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875f85f-0a9f-4555-b7c4-0ae0245bfdbd None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:37.303715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875f85f-0a9f-4555-b7c4-0ae0245bfdbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:37.371535 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4875f85f-0a9f-4555-b7c4-0ae0245bfdbd None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:59:37.372786 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4875f85f-0a9f-4555-b7c4-0ae0245bfdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:37.469418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 743/1472] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:59:37.771907 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3371d-e1fc-47c6-b554-ecda1f2b6ae2 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:37.772196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3371d-e1fc-47c6-b554-ecda1f2b6ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:37.772472 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3371d-e1fc-47c6-b554-ecda1f2b6ae2 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:37.773819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 730/1473] 10.222.0.42 () {54 vars in 966 bytes} [Wed Sep 14 00:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:59:37.778926 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09711b52-053f-4f54-8177-ff116b27c62d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:37.779216 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09711b52-053f-4f54-8177-ff116b27c62d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:37.779493 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09711b52-053f-4f54-8177-ff116b27c62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:37.857926 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-09711b52-053f-4f54-8177-ff116b27c62d None None] MFA Rules not processed for user `89ff357209194276bc079d0c05898747`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:59:37.859098 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-09711b52-053f-4f54-8177-ff116b27c62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:37.880455 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:37.936989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 744/1474] 10.222.0.42 () {58 vars in 1063 bytes} [Wed Sep 14 00:59:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:59:37.944679 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3371d-e1fc-47c6-b554-ecda1f2b6ae2 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:59:37.946212 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0cb3371d-e1fc-47c6-b554-ecda1f2b6ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:38.049626 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3371d-e1fc-47c6-b554-ecda1f2b6ae2 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:59:38.122932 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf3d283-9034-4354-afa9-f8bb35c93998 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89ff357209194276bc079d0c05898747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:59:38.123459 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3d283-9034-4354-afa9-f8bb35c93998 service glance] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:38.123765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3d283-9034-4354-afa9-f8bb35c93998 service glance] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:38.124043 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3d283-9034-4354-afa9-f8bb35c93998 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:38.125618 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-cbf3d283-9034-4354-afa9-f8bb35c93998 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:38.179271 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:38.233748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf3d283-9034-4354-afa9-f8bb35c93998 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:59:38.235187 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf3d283-9034-4354-afa9-f8bb35c93998 service glance] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:59:38.254992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 731/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:59:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:59:40.487272 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb50893-b25c-4ce7-890e-992ea93803be None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:40.487807 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb50893-b25c-4ce7-890e-992ea93803be None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:40.487923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb50893-b25c-4ce7-890e-992ea93803be None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:40.489184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 745/1476] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:59:40.494507 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ca0d2-5635-456a-8136-fb2437c27489 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:40.494948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ca0d2-5635-456a-8136-fb2437c27489 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:40.495086 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ca0d2-5635-456a-8136-fb2437c27489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:40.561638 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-bf7ca0d2-5635-456a-8136-fb2437c27489 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:59:40.562855 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bf7ca0d2-5635-456a-8136-fb2437c27489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:40.663989 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 732/1477] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:59:40.945350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb50893-b25c-4ce7-890e-992ea93803be None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:59:40.946945 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dbb50893-b25c-4ce7-890e-992ea93803be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:41.190550 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb50893-b25c-4ce7-890e-992ea93803be None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:59:41.239775 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:41.285169 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b2d25-dee1-47c9-bcda-45e74106eb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:59:41.285774 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b2d25-dee1-47c9-bcda-45e74106eb29 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:41.286051 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b2d25-dee1-47c9-bcda-45e74106eb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:41.286322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b2d25-dee1-47c9-bcda-45e74106eb29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:41.286960 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b2d25-dee1-47c9-bcda-45e74106eb29 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:59:41.288721 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: WARNING py.warnings [None req-ac1b2d25-dee1-47c9-bcda-45e74106eb29 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 00:59:41.288721 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: warnings.warn(msg) Sep 14 00:59:41.288721 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: Sep 14 00:59:41.289846 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b2d25-dee1-47c9-bcda-45e74106eb29 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:59:41.309299 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 746/1478] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 00:59:40 2022] GET /identity/v3/projects => generated 2553 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 00:59:43.490023 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55393335-2c4b-4a9d-ba7c-52818f2f6f3b None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:43.490479 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55393335-2c4b-4a9d-ba7c-52818f2f6f3b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:43.490594 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55393335-2c4b-4a9d-ba7c-52818f2f6f3b None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:43.491943 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 733/1479] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:59:43.497230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4d935-fe17-419b-bcb5-26fccd9783d8 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:43.497635 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4d935-fe17-419b-bcb5-26fccd9783d8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:43.497878 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4d935-fe17-419b-bcb5-26fccd9783d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:43.564293 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ffa4d935-fe17-419b-bcb5-26fccd9783d8 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:59:43.565502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ffa4d935-fe17-419b-bcb5-26fccd9783d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:43.663582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 747/1480] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:59:43.962875 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55393335-2c4b-4a9d-ba7c-52818f2f6f3b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:59:43.965132 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-55393335-2c4b-4a9d-ba7c-52818f2f6f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:43.975180 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:44.112322 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55393335-2c4b-4a9d-ba7c-52818f2f6f3b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:59:44.191286 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:44.225080 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f43da-aa07-444c-ac55-994d0bd07119 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:59:44.225619 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f43da-aa07-444c-ac55-994d0bd07119 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:44.225901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f43da-aa07-444c-ac55-994d0bd07119 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:44.226176 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f43da-aa07-444c-ac55-994d0bd07119 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:44.227841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-035f43da-aa07-444c-ac55-994d0bd07119 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:44.319269 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035f43da-aa07-444c-ac55-994d0bd07119 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:59:44.321009 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035f43da-aa07-444c-ac55-994d0bd07119 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:59:44.339519 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 734/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:59:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:59:46.690849 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647d020-4c74-4a25-828f-00a97601e090 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:46.691324 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647d020-4c74-4a25-828f-00a97601e090 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:46.691421 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647d020-4c74-4a25-828f-00a97601e090 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:46.692765 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 748/1482] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:59:46.698097 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b806c4-ed72-4acc-a330-cc660c31274b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:46.698774 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b806c4-ed72-4acc-a330-cc660c31274b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:46.698957 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b806c4-ed72-4acc-a330-cc660c31274b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:46.764909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-f2b806c4-ed72-4acc-a330-cc660c31274b None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:59:46.766043 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f2b806c4-ed72-4acc-a330-cc660c31274b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:46.812817 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:46.862618 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 735/1483] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:59:47.155364 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647d020-4c74-4a25-828f-00a97601e090 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:59:47.156957 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3647d020-4c74-4a25-828f-00a97601e090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:47.180102 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:47.262287 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647d020-4c74-4a25-828f-00a97601e090 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:59:47.335538 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d320a4f-c28d-4c12-92c1-e6051863749c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:59:47.336175 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d320a4f-c28d-4c12-92c1-e6051863749c service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:47.336456 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d320a4f-c28d-4c12-92c1-e6051863749c service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:47.336730 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d320a4f-c28d-4c12-92c1-e6051863749c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:47.338462 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2d320a4f-c28d-4c12-92c1-e6051863749c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:47.423338 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:47.458662 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d320a4f-c28d-4c12-92c1-e6051863749c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:59:47.460142 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d320a4f-c28d-4c12-92c1-e6051863749c service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:59:47.478190 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 749/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:59:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:59:49.699566 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af0631-42c3-4086-98e0-8505b35a1e67 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:49.700129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af0631-42c3-4086-98e0-8505b35a1e67 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:49.700129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af0631-42c3-4086-98e0-8505b35a1e67 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:49.701499 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 736/1485] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:59:49.706858 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92be245-c82d-4403-ad7f-5f5832c38d78 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:49.707306 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92be245-c82d-4403-ad7f-5f5832c38d78 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:49.707414 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92be245-c82d-4403-ad7f-5f5832c38d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:49.773205 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b92be245-c82d-4403-ad7f-5f5832c38d78 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:59:49.774334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b92be245-c82d-4403-ad7f-5f5832c38d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:49.869279 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 750/1486] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:59:50.158783 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af0631-42c3-4086-98e0-8505b35a1e67 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:59:50.160345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-67af0631-42c3-4086-98e0-8505b35a1e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:50.264340 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af0631-42c3-4086-98e0-8505b35a1e67 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:59:50.338404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:50.340510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de4650-379e-4e10-8d2a-bc40040faf8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:59:50.341204 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de4650-379e-4e10-8d2a-bc40040faf8a service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:50.341680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de4650-379e-4e10-8d2a-bc40040faf8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:50.342101 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de4650-379e-4e10-8d2a-bc40040faf8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:50.343974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-45de4650-379e-4e10-8d2a-bc40040faf8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:50.452326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de4650-379e-4e10-8d2a-bc40040faf8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:59:50.453768 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de4650-379e-4e10-8d2a-bc40040faf8a service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:59:50.472179 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 737/1487] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:59:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:59:52.740296 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb27c77-965a-4315-a5ad-5c5529004794 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:52.740807 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb27c77-965a-4315-a5ad-5c5529004794 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:52.740807 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb27c77-965a-4315-a5ad-5c5529004794 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:52.742114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 751/1488] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:59:52.747994 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b4601-802f-460b-9638-9e699387fb67 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:52.748475 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b4601-802f-460b-9638-9e699387fb67 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:52.748584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b4601-802f-460b-9638-9e699387fb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:52.814489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-0d7b4601-802f-460b-9638-9e699387fb67 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:59:52.815758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0d7b4601-802f-460b-9638-9e699387fb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:52.868221 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:52.912831 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 738/1489] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:59:53.204732 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb27c77-965a-4315-a5ad-5c5529004794 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:59:53.206264 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9eb27c77-965a-4315-a5ad-5c5529004794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:53.242971 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:53.311643 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb27c77-965a-4315-a5ad-5c5529004794 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:59:53.384922 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c14a6-00d3-460b-9556-c704485991b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:59:53.385528 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c14a6-00d3-460b-9556-c704485991b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:53.386351 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c14a6-00d3-460b-9556-c704485991b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:53.386588 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c14a6-00d3-460b-9556-c704485991b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:53.388458 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c41c14a6-00d3-460b-9556-c704485991b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:53.471083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:53.499426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c14a6-00d3-460b-9556-c704485991b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:59:53.500893 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c14a6-00d3-460b-9556-c704485991b4 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:59:53.521707 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 752/1490] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:59:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 00:59:55.901251 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d1dd3-8501-4240-9444-561b2e866045 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:55.901802 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d1dd3-8501-4240-9444-561b2e866045 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:55.901802 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d1dd3-8501-4240-9444-561b2e866045 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:55.903044 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 739/1491] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 00:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 00:59:55.910517 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7735e-6226-4900-b65c-09ee172e420d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:55.910961 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7735e-6226-4900-b65c-09ee172e420d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:55.911090 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7735e-6226-4900-b65c-09ee172e420d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:55.977062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c0d7735e-6226-4900-b65c-09ee172e420d None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 00:59:55.978261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c0d7735e-6226-4900-b65c-09ee172e420d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:56.075313 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 753/1492] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 00:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 00:59:56.367373 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d1dd3-8501-4240-9444-561b2e866045 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 00:59:56.368953 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e28d1dd3-8501-4240-9444-561b2e866045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:56.477576 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d1dd3-8501-4240-9444-561b2e866045 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 00:59:56.554421 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb1966-ee85-4869-aa04-427d319786bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 00:59:56.555054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb1966-ee85-4869-aa04-427d319786bc service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 00:59:56.555347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb1966-ee85-4869-aa04-427d319786bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 00:59:56.555629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb1966-ee85-4869-aa04-427d319786bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 00:59:56.557224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e1fb1966-ee85-4869-aa04-427d319786bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 00:59:56.566419 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 00:59:56.668625 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb1966-ee85-4869-aa04-427d319786bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 00:59:56.670126 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb1966-ee85-4869-aa04-427d319786bc service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 00:59:56.689130 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 740/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 00:59:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:00:06.074010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=82 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:00:06.686848 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=50 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:01:09.918481 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447e395-36b0-416d-abe6-56398f389a60 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:01:09.919106 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447e395-36b0-416d-abe6-56398f389a60 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:01:09.919106 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447e395-36b0-416d-abe6-56398f389a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:01:09.993458 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b447e395-36b0-416d-abe6-56398f389a60 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:01:09.994764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b447e395-36b0-416d-abe6-56398f389a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:01:10.093507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 754/1494] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:01:10.101442 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22de4e-170a-4dcc-97b5-fd52bdc2e20e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:01:10.101928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22de4e-170a-4dcc-97b5-fd52bdc2e20e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:01:10.102035 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22de4e-170a-4dcc-97b5-fd52bdc2e20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:01:10.178869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-5a22de4e-170a-4dcc-97b5-fd52bdc2e20e None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:01:10.180050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5a22de4e-170a-4dcc-97b5-fd52bdc2e20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:01:10.276874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 741/1495] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:01:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:01:10.281658 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447e395-36b0-416d-abe6-56398f389a60 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:01:10.282853 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b447e395-36b0-416d-abe6-56398f389a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:01:10.294242 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447e395-36b0-416d-abe6-56398f389a60 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:01:10.354758 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:01:10.389677 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8d049-ee31-4b45-804d-20abbc26ac9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:01:10.390282 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8d049-ee31-4b45-804d-20abbc26ac9f admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:01:10.390563 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8d049-ee31-4b45-804d-20abbc26ac9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:01:10.390837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8d049-ee31-4b45-804d-20abbc26ac9f admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:01:10.391499 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8d049-ee31-4b45-804d-20abbc26ac9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:01:10.391820 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8d049-ee31-4b45-804d-20abbc26ac9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:01:10.392840 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: WARNING py.warnings [None req-9ff8d049-ee31-4b45-804d-20abbc26ac9f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 14 01:01:10.392840 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: warnings.warn(msg) Sep 14 01:01:10.392840 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: Sep 14 01:01:10.393731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8d049-ee31-4b45-804d-20abbc26ac9f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:01:10.404961 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 755/1496] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:01:10.413390 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22de4e-170a-4dcc-97b5-fd52bdc2e20e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:01:10.422606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22de4e-170a-4dcc-97b5-fd52bdc2e20e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:01:10.485014 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:01:10.520258 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d5689-7f0a-4642-9870-685264998b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:01:10.520870 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d5689-7f0a-4642-9870-685264998b2a admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:01:10.521148 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d5689-7f0a-4642-9870-685264998b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:01:10.521418 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d5689-7f0a-4642-9870-685264998b2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:01:10.522047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d5689-7f0a-4642-9870-685264998b2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:01:10.523866 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: WARNING py.warnings [None req-bc5d5689-7f0a-4642-9870-685264998b2a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 01:01:10.523866 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: warnings.warn(msg) Sep 14 01:01:10.523866 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: Sep 14 01:01:10.525109 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d5689-7f0a-4642-9870-685264998b2a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:01:10.581825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 742/1497] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:01:10 2022] POST /identity/v3/projects => generated 377 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:01:10.585998 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8d049-ee31-4b45-804d-20abbc26ac9f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:01:10.587528 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9ff8d049-ee31-4b45-804d-20abbc26ac9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:01:10.621121 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8d049-ee31-4b45-804d-20abbc26ac9f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:01:10.718666 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda9383-ec22-46e0-8ea3-004c3854b7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:01:10.719273 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda9383-ec22-46e0-8ea3-004c3854b7e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:01:10.719630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda9383-ec22-46e0-8ea3-004c3854b7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:01:10.719858 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda9383-ec22-46e0-8ea3-004c3854b7e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:01:10.720491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda9383-ec22-46e0-8ea3-004c3854b7e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:01:10.721500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: WARNING py.warnings [None req-8fda9383-ec22-46e0-8ea3-004c3854b7e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 01:01:10.721500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: warnings.warn(msg) Sep 14 01:01:10.721500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: Sep 14 01:01:10.722704 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda9383-ec22-46e0-8ea3-004c3854b7e8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:01:10.765351 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 756/1498] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:01:10 2022] POST /identity/v3/users => generated 289 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:01:10.769859 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d5689-7f0a-4642-9870-685264998b2a admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:01:10.771425 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bc5d5689-7f0a-4642-9870-685264998b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:01:10.833383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d5689-7f0a-4642-9870-685264998b2a admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:01:10.933783 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:01:10.945895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a09b2-8049-4a04-bb1e-fb113a9bf149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:01:10.946548 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a09b2-8049-4a04-bb1e-fb113a9bf149 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:01:10.946889 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a09b2-8049-4a04-bb1e-fb113a9bf149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:01:10.947193 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a09b2-8049-4a04-bb1e-fb113a9bf149 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:01:10.948399 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a09b2-8049-4a04-bb1e-fb113a9bf149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:01:10.949462 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: WARNING py.warnings [None req-452a09b2-8049-4a04-bb1e-fb113a9bf149 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 14 01:01:10.949462 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: warnings.warn(msg) Sep 14 01:01:10.949462 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: Sep 14 01:01:10.950530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a09b2-8049-4a04-bb1e-fb113a9bf149 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:01:10.964694 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 743/1499] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:01:10 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:01:10.968860 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda9383-ec22-46e0-8ea3-004c3854b7e8 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:01:10.970443 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8fda9383-ec22-46e0-8ea3-004c3854b7e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:01:10.982664 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda9383-ec22-46e0-8ea3-004c3854b7e8 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:01:11.008167 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:01:11.080507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6fcc23-98c6-4efd-9c8e-e5e141141e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:01:11.081042 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fcc23-98c6-4efd-9c8e-e5e141141e25 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:01:11.081324 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fcc23-98c6-4efd-9c8e-e5e141141e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:01:11.081600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fcc23-98c6-4efd-9c8e-e5e141141e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:01:11.082241 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6fcc23-98c6-4efd-9c8e-e5e141141e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:01:11.083282 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: WARNING py.warnings [None req-ba6fcc23-98c6-4efd-9c8e-e5e141141e25 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 14 01:01:11.083282 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: warnings.warn(msg) Sep 14 01:01:11.083282 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: Sep 14 01:01:11.084204 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6fcc23-98c6-4efd-9c8e-e5e141141e25 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:01:11.097140 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 757/1500] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:01:10 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:01:11.101358 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a09b2-8049-4a04-bb1e-fb113a9bf149 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:01:11.102869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-452a09b2-8049-4a04-bb1e-fb113a9bf149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:01:11.136861 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a09b2-8049-4a04-bb1e-fb113a9bf149 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:01:11.232291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663f65c-181f-4991-b4e2-7cd14145617e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:01:11.232954 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663f65c-181f-4991-b4e2-7cd14145617e admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:01:11.233238 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663f65c-181f-4991-b4e2-7cd14145617e admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:01:11.233518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663f65c-181f-4991-b4e2-7cd14145617e admin admin] PATH_INFO: `/v3/projects/38b4ff1e70c347b4ac19492b4ec26b51/users/e005484857a447d6977e5d18163b8cca/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:01:11.255958 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5663f65c-181f-4991-b4e2-7cd14145617e admin admin] RBAC: Authorizing `identity:create_grant(project_id=38b4ff1e70c347b4ac19492b4ec26b51, user_id=e005484857a447d6977e5d18163b8cca, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:01:11.257006 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: WARNING py.warnings [None req-5663f65c-181f-4991-b4e2-7cd14145617e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 01:01:11.257006 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: warnings.warn(msg) Sep 14 01:01:11.257006 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: Sep 14 01:01:11.258952 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5663f65c-181f-4991-b4e2-7cd14145617e admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:01:11.283733 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 744/1501] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:01:11 2022] PUT /identity/v3/projects/38b4ff1e70c347b4ac19492b4ec26b51/users/e005484857a447d6977e5d18163b8cca/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:01:11.296431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d26a1-bc4a-420f-b926-7cac11a52355 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:01:11.296755 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d26a1-bc4a-420f-b926-7cac11a52355 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:01:11.297039 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d26a1-bc4a-420f-b926-7cac11a52355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:01:11.352905 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0d4d26a1-bc4a-420f-b926-7cac11a52355 None None] MFA Rules not processed for user `e005484857a447d6977e5d18163b8cca`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:01:11.354616 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0d4d26a1-bc4a-420f-b926-7cac11a52355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:01:11.432514 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 758/1502] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:01:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:01:11.445611 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572e37a-788c-42d4-897e-3ce50bac8bb6 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:01:11.445903 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572e37a-788c-42d4-897e-3ce50bac8bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:01:11.446182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572e37a-788c-42d4-897e-3ce50bac8bb6 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:01:11.448122 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 745/1503] 10.222.0.42 () {54 vars in 964 bytes} [Wed Sep 14 01:01:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:01:11.452803 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8097a-ace2-4701-8d7d-9b04cb925eb4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:01:11.453092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8097a-ace2-4701-8d7d-9b04cb925eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:01:11.453365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8097a-ace2-4701-8d7d-9b04cb925eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:01:11.490674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:01:11.536443 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b4d8097a-ace2-4701-8d7d-9b04cb925eb4 None None] MFA Rules not processed for user `9373aa297439487691fae325c0245bb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:01:11.537684 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b4d8097a-ace2-4701-8d7d-9b04cb925eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:01:11.612267 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 759/1504] 10.222.0.42 () {58 vars in 1061 bytes} [Wed Sep 14 01:01:11 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:01:11.622069 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8572e37a-788c-42d4-897e-3ce50bac8bb6 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:01:11.623596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-8572e37a-788c-42d4-897e-3ce50bac8bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:01:11.674049 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:01:11.728243 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8572e37a-788c-42d4-897e-3ce50bac8bb6 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:01:11.802836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca6a342-28c8-44d5-99c8-8af3024a1dd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:01:11.803429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6a342-28c8-44d5-99c8-8af3024a1dd1 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:01:11.803737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6a342-28c8-44d5-99c8-8af3024a1dd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:01:11.804006 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6a342-28c8-44d5-99c8-8af3024a1dd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:01:11.805576 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bca6a342-28c8-44d5-99c8-8af3024a1dd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:01:12.023186 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca6a342-28c8-44d5-99c8-8af3024a1dd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:01:12.024637 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca6a342-28c8-44d5-99c8-8af3024a1dd1 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:01:12.028874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:01:12.044892 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 746/1505] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:01:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:01:12.091117 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052fa6c-2d6c-4d63-9b3c-c765fff7c1f6 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:01:12.091576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052fa6c-2d6c-4d63-9b3c-c765fff7c1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:01:12.091709 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052fa6c-2d6c-4d63-9b3c-c765fff7c1f6 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:01:12.093022 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 760/1506] 10.222.0.42 () {52 vars in 830 bytes} [Wed Sep 14 01:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:01:12.096669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca6a342-28c8-44d5-99c8-8af3024a1dd1 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:01:12.097896 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bca6a342-28c8-44d5-99c8-8af3024a1dd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:01:12.198027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca6a342-28c8-44d5-99c8-8af3024a1dd1 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:01:12.290128 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e2948-39f7-4559-ac06-2c237d5058a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:01:12.290917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e2948-39f7-4559-ac06-2c237d5058a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:01:12.291202 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e2948-39f7-4559-ac06-2c237d5058a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:01:12.291477 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e2948-39f7-4559-ac06-2c237d5058a1 admin admin] PATH_INFO: `/v3/users/e005484857a447d6977e5d18163b8cca` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:01:12.304295 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e2948-39f7-4559-ac06-2c237d5058a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e005484857a447d6977e5d18163b8cca)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:01:12.305326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: WARNING py.warnings [None req-de8e2948-39f7-4559-ac06-2c237d5058a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 01:01:12.305326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: warnings.warn(msg) Sep 14 01:01:12.305326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: Sep 14 01:01:12.306600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e2948-39f7-4559-ac06-2c237d5058a1 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:01:12.316141 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:01:12.373329 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-de8e2948-39f7-4559-ac06-2c237d5058a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e005484857a447d6977e5d18163b8cca'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:01:12.373857 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-de8e2948-39f7-4559-ac06-2c237d5058a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e005484857a447d6977e5d18163b8cca'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:01:12.382892 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-de8e2948-39f7-4559-ac06-2c237d5058a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e005484857a447d6977e5d18163b8cca'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:01:12.398930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-de8e2948-39f7-4559-ac06-2c237d5058a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e005484857a447d6977e5d18163b8cca'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:01:12.399371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-de8e2948-39f7-4559-ac06-2c237d5058a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e005484857a447d6977e5d18163b8cca'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:01:12.427519 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 747/1507] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:01:12 2022] DELETE /identity/v3/users/e005484857a447d6977e5d18163b8cca => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:01:12.432421 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b33789-7834-4764-8b46-9730f3788568 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:01:12.432728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b33789-7834-4764-8b46-9730f3788568 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:01:12.432989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b33789-7834-4764-8b46-9730f3788568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:01:12.494021 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-66b33789-7834-4764-8b46-9730f3788568 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:01:12.495274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-66b33789-7834-4764-8b46-9730f3788568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:01:12.587907 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:01:12.600585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 761/1508] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:01:12.610249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e2948-39f7-4559-ac06-2c237d5058a1 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:01:12.611466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-de8e2948-39f7-4559-ac06-2c237d5058a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:01:12.711436 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e2948-39f7-4559-ac06-2c237d5058a1 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:01:12.767847 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:01:12.788899 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa534b-94c4-4563-8889-7a276f3c6a64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:01:12.789520 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa534b-94c4-4563-8889-7a276f3c6a64 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:01:12.789807 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa534b-94c4-4563-8889-7a276f3c6a64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:01:12.790084 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa534b-94c4-4563-8889-7a276f3c6a64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:01:12.792089 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a9aa534b-94c4-4563-8889-7a276f3c6a64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:01:12.902177 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa534b-94c4-4563-8889-7a276f3c6a64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:01:12.903611 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa534b-94c4-4563-8889-7a276f3c6a64 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:01:12.921928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 748/1509] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:01:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:01:13.281244 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b33789-7834-4764-8b46-9730f3788568 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:01:13.282520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-66b33789-7834-4764-8b46-9730f3788568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:01:13.294608 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b33789-7834-4764-8b46-9730f3788568 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:01:13.393979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523de58f-28ef-4e58-a628-5a3653af9309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:01:13.394615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523de58f-28ef-4e58-a628-5a3653af9309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:01:13.394896 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523de58f-28ef-4e58-a628-5a3653af9309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:01:13.395171 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523de58f-28ef-4e58-a628-5a3653af9309 admin admin] PATH_INFO: `/v3/projects/38b4ff1e70c347b4ac19492b4ec26b51` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:01:13.405550 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523de58f-28ef-4e58-a628-5a3653af9309 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38b4ff1e70c347b4ac19492b4ec26b51)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:01:13.406523 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: WARNING py.warnings [None req-523de58f-28ef-4e58-a628-5a3653af9309 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 01:01:13.406523 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: warnings.warn(msg) Sep 14 01:01:13.406523 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: Sep 14 01:01:13.407749 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523de58f-28ef-4e58-a628-5a3653af9309 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:01:13.466392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-523de58f-28ef-4e58-a628-5a3653af9309 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38b4ff1e70c347b4ac19492b4ec26b51', 'request_id': 'req-523de58f-28ef-4e58-a628-5a3653af9309'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:01:13.480140 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-523de58f-28ef-4e58-a628-5a3653af9309 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38b4ff1e70c347b4ac19492b4ec26b51', 'request_id': 'req-523de58f-28ef-4e58-a628-5a3653af9309'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:01:13.484476 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-523de58f-28ef-4e58-a628-5a3653af9309 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38b4ff1e70c347b4ac19492b4ec26b51', 'request_id': 'req-523de58f-28ef-4e58-a628-5a3653af9309'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:01:13.515991 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-523de58f-28ef-4e58-a628-5a3653af9309 admin admin] The token cache is being invalidate because project 38b4ff1e70c347b4ac19492b4ec26b51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:01:13.516274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-523de58f-28ef-4e58-a628-5a3653af9309 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:01:13.516697 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-523de58f-28ef-4e58-a628-5a3653af9309 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:01:13.518093 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 762/1510] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:01:13 2022] DELETE /identity/v3/projects/38b4ff1e70c347b4ac19492b4ec26b51 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:01:22.919685 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=62 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:01:23.513701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:02:40.284260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814f41d-c263-47cb-9edc-a2d134642f6b None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:02:40.284752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814f41d-c263-47cb-9edc-a2d134642f6b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:02:40.284853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814f41d-c263-47cb-9edc-a2d134642f6b None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:02:40.286232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 749/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 01:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:02:40.291192 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48234d-aedf-4e74-bb0a-0bcf6cdcc9e0 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:02:40.291654 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48234d-aedf-4e74-bb0a-0bcf6cdcc9e0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:02:40.291765 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48234d-aedf-4e74-bb0a-0bcf6cdcc9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:02:40.373078 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-dd48234d-aedf-4e74-bb0a-0bcf6cdcc9e0 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:02:40.374196 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dd48234d-aedf-4e74-bb0a-0bcf6cdcc9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:02:40.476836 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 763/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 01:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:02:40.632423 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814f41d-c263-47cb-9edc-a2d134642f6b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:02:40.634540 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a814f41d-c263-47cb-9edc-a2d134642f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:02:40.757376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814f41d-c263-47cb-9edc-a2d134642f6b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:02:40.764246 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c2346c-bf76-49a3-a04a-3607045a5288 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:02:40.765115 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2346c-bf76-49a3-a04a-3607045a5288 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:02:40.765398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2346c-bf76-49a3-a04a-3607045a5288 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:02:40.766084 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2346c-bf76-49a3-a04a-3607045a5288 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:02:40.774291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c2346c-bf76-49a3-a04a-3607045a5288 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:02:40.775923 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c2346c-bf76-49a3-a04a-3607045a5288 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:02:40.786089 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 750/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:02:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:02:41.871914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad72308-3b03-4309-9ddb-a4a1d2400e36 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:02:41.872462 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad72308-3b03-4309-9ddb-a4a1d2400e36 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:02:41.872462 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad72308-3b03-4309-9ddb-a4a1d2400e36 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:02:41.873716 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 764/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 01:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:02:41.879911 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a5d85-7dae-4c42-bf03-a818ae15dce8 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:02:41.880365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a5d85-7dae-4c42-bf03-a818ae15dce8 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:02:41.880502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a5d85-7dae-4c42-bf03-a818ae15dce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:02:41.900002 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-8f1a5d85-7dae-4c42-bf03-a818ae15dce8 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:02:41.901194 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-8f1a5d85-7dae-4c42-bf03-a818ae15dce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:02:41.909592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 751/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 01:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:02:42.067907 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad72308-3b03-4309-9ddb-a4a1d2400e36 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:02:42.082924 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad72308-3b03-4309-9ddb-a4a1d2400e36 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:02:42.088385 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1547a0f6-7f7d-4e3c-afb8-576c935bb64e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:02:42.088983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547a0f6-7f7d-4e3c-afb8-576c935bb64e service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:02:42.089266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547a0f6-7f7d-4e3c-afb8-576c935bb64e service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:02:42.090092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547a0f6-7f7d-4e3c-afb8-576c935bb64e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:02:42.098856 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1547a0f6-7f7d-4e3c-afb8-576c935bb64e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:02:42.100568 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1547a0f6-7f7d-4e3c-afb8-576c935bb64e service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:02:42.110627 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 765/1516] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:02:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:02:42.192102 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabeb90-86a3-4e18-a56d-e2a2b2f79d9d None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:02:42.192392 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabeb90-86a3-4e18-a56d-e2a2b2f79d9d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:02:42.192666 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabeb90-86a3-4e18-a56d-e2a2b2f79d9d None None] PATH_INFO: `/v3` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:02:42.193962 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 752/1517] 10.222.0.44 () {54 vars in 925 bytes} [Wed Sep 14 01:02:42 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:02:42.201410 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e9dd6-fd0f-4a38-91e8-a959dbb589fd None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:02:42.201724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e9dd6-fd0f-4a38-91e8-a959dbb589fd None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:02:42.202002 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e9dd6-fd0f-4a38-91e8-a959dbb589fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:02:42.266907 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-707e9dd6-fd0f-4a38-91e8-a959dbb589fd None None] MFA Rules not processed for user `18798838de284c84b5b838b3530a2c43`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:02:42.268254 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-707e9dd6-fd0f-4a38-91e8-a959dbb589fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:02:42.286053 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:02:42.343089 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 766/1518] 10.222.0.44 () {58 vars in 1015 bytes} [Wed Sep 14 01:02:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:02:51.353872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fec55-caca-44b9-af0e-da8134b5017e None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:02:51.354363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fec55-caca-44b9-af0e-da8134b5017e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:02:51.354474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fec55-caca-44b9-af0e-da8134b5017e None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:02:51.355975 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 753/1519] 10.222.0.42 () {54 vars in 961 bytes} [Wed Sep 14 01:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:02:51.361965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a6900-17f9-46cc-bbf9-2ed8da809e4b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:02:51.362423 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a6900-17f9-46cc-bbf9-2ed8da809e4b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:02:51.362527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a6900-17f9-46cc-bbf9-2ed8da809e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:02:51.381828 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-542a6900-17f9-46cc-bbf9-2ed8da809e4b None None] MFA Rules not processed for user `18798838de284c84b5b838b3530a2c43`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:02:51.383018 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-542a6900-17f9-46cc-bbf9-2ed8da809e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:02:51.392244 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 767/1520] 10.222.0.42 () {58 vars in 1058 bytes} [Wed Sep 14 01:02:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:02:51.399655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9fec55-caca-44b9-af0e-da8134b5017e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:02:51.413694 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9fec55-caca-44b9-af0e-da8134b5017e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:02:51.418894 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55733039-3bee-4527-9613-0e1ae6fedead service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18798838de284c84b5b838b3530a2c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:02:51.419481 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55733039-3bee-4527-9613-0e1ae6fedead service placement] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:02:51.419788 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55733039-3bee-4527-9613-0e1ae6fedead service placement] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:02:51.420058 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55733039-3bee-4527-9613-0e1ae6fedead service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:02:51.428100 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55733039-3bee-4527-9613-0e1ae6fedead service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:02:51.429661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55733039-3bee-4527-9613-0e1ae6fedead service placement] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:02:51.439871 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 754/1521] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:02:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:02:53.599012 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb4635-4a79-4bbb-b640-71b30292d729 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:02:53.599683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb4635-4a79-4bbb-b640-71b30292d729 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:02:53.599683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb4635-4a79-4bbb-b640-71b30292d729 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:02:53.600926 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 768/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 01:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:02:53.607998 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268df24-a6d2-42a7-bdab-4513f93d174d None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:02:53.608468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268df24-a6d2-42a7-bdab-4513f93d174d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:02:53.608574 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268df24-a6d2-42a7-bdab-4513f93d174d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:02:53.628211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-c268df24-a6d2-42a7-bdab-4513f93d174d None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:02:53.629397 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c268df24-a6d2-42a7-bdab-4513f93d174d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:02:53.637257 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 755/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 01:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:02:53.792215 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb4635-4a79-4bbb-b640-71b30292d729 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:02:53.807250 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb4635-4a79-4bbb-b640-71b30292d729 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:02:53.812761 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d8580a-ddc0-4bfa-9705-b32f703b7097 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:02:53.813358 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8580a-ddc0-4bfa-9705-b32f703b7097 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:02:53.813918 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8580a-ddc0-4bfa-9705-b32f703b7097 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:02:53.814197 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8580a-ddc0-4bfa-9705-b32f703b7097 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:02:53.822178 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d8580a-ddc0-4bfa-9705-b32f703b7097 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:02:53.823902 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d8580a-ddc0-4bfa-9705-b32f703b7097 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:02:53.833277 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 769/1524] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:02:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:02:54.919810 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f9363-35af-4257-94c7-c74835104a75 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:02:54.920386 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f9363-35af-4257-94c7-c74835104a75 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:02:54.920386 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f9363-35af-4257-94c7-c74835104a75 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:02:54.921692 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 756/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 01:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:02:54.927730 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d15ad-46f7-4a15-af48-9f6859fe608d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:02:54.928167 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d15ad-46f7-4a15-af48-9f6859fe608d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:02:54.928291 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d15ad-46f7-4a15-af48-9f6859fe608d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:02:54.947075 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a40d15ad-46f7-4a15-af48-9f6859fe608d None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:02:54.948327 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a40d15ad-46f7-4a15-af48-9f6859fe608d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:02:54.956726 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 770/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 01:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:02:55.111620 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f9363-35af-4257-94c7-c74835104a75 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:02:55.130736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f9363-35af-4257-94c7-c74835104a75 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:02:55.136420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2954a-ba36-49e4-b813-24479b721879 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:02:55.137217 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2954a-ba36-49e4-b813-24479b721879 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:02:55.137501 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2954a-ba36-49e4-b813-24479b721879 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:02:55.137772 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2954a-ba36-49e4-b813-24479b721879 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:02:55.145904 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2954a-ba36-49e4-b813-24479b721879 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:02:55.147481 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2954a-ba36-49e4-b813-24479b721879 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:02:55.157638 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 757/1527] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:02:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:02:59.140591 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5684f7-1a6c-475d-af83-be590a893509 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:02:59.141127 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5684f7-1a6c-475d-af83-be590a893509 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:02:59.141127 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5684f7-1a6c-475d-af83-be590a893509 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:02:59.142365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 771/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 01:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:02:59.147674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec9b5f-4ade-4459-8fb8-53517689ec1c None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:02:59.148106 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec9b5f-4ade-4459-8fb8-53517689ec1c None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:02:59.148256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec9b5f-4ade-4459-8fb8-53517689ec1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:02:59.168710 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e3ec9b5f-4ade-4459-8fb8-53517689ec1c None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:02:59.169901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e3ec9b5f-4ade-4459-8fb8-53517689ec1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:02:59.177637 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 758/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 01:02:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:02:59.466597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5684f7-1a6c-475d-af83-be590a893509 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:02:59.468347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bf5684f7-1a6c-475d-af83-be590a893509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:02:59.549159 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5684f7-1a6c-475d-af83-be590a893509 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:02:59.552678 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc6531-bfe2-4180-8e4c-b7dd74820a72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:02:59.553276 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc6531-bfe2-4180-8e4c-b7dd74820a72 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:02:59.553554 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc6531-bfe2-4180-8e4c-b7dd74820a72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:02:59.553830 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc6531-bfe2-4180-8e4c-b7dd74820a72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:02:59.563007 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc6531-bfe2-4180-8e4c-b7dd74820a72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:02:59.564500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc6531-bfe2-4180-8e4c-b7dd74820a72 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:02:59.574569 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 772/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:02:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:03:01.733799 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6fff32-d737-44c6-97f8-d2247c713d94 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:03:01.734315 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6fff32-d737-44c6-97f8-d2247c713d94 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:03:01.734315 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6fff32-d737-44c6-97f8-d2247c713d94 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:03:01.735650 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 759/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 01:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:03:01.740889 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7038278-4a4c-4c98-b7a3-35a643175a37 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:03:01.741358 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7038278-4a4c-4c98-b7a3-35a643175a37 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:03:01.741470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7038278-4a4c-4c98-b7a3-35a643175a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:03:01.761498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d7038278-4a4c-4c98-b7a3-35a643175a37 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:03:01.762685 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d7038278-4a4c-4c98-b7a3-35a643175a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:03:01.770626 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 773/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 01:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:03:02.059518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6fff32-d737-44c6-97f8-d2247c713d94 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:03:02.061286 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bc6fff32-d737-44c6-97f8-d2247c713d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:03:02.073979 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6fff32-d737-44c6-97f8-d2247c713d94 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:03:02.077448 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b09f29-193f-4d85-9742-9fd6d229b775 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:03:02.078038 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b09f29-193f-4d85-9742-9fd6d229b775 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:03:02.078311 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b09f29-193f-4d85-9742-9fd6d229b775 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:03:02.078580 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b09f29-193f-4d85-9742-9fd6d229b775 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:03:02.087224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b09f29-193f-4d85-9742-9fd6d229b775 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:03:02.088760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b09f29-193f-4d85-9742-9fd6d229b775 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:03:02.098466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 760/1533] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:03:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:03:05.773714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c3da9-f096-47e3-b205-51d6184c5778 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:03:05.774307 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c3da9-f096-47e3-b205-51d6184c5778 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:03:05.774307 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c3da9-f096-47e3-b205-51d6184c5778 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:03:05.775737 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 774/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 01:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:03:05.781717 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc038d40-1c63-4ff6-9a58-386a93e48572 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:03:05.782291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc038d40-1c63-4ff6-9a58-386a93e48572 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:03:05.782291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc038d40-1c63-4ff6-9a58-386a93e48572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:03:05.789924 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=81 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:03:05.803083 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-cc038d40-1c63-4ff6-9a58-386a93e48572 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:03:05.804333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-cc038d40-1c63-4ff6-9a58-386a93e48572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:03:05.812328 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 761/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 01:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:03:06.122898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c3da9-f096-47e3-b205-51d6184c5778 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:03:06.124634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bc2c3da9-f096-47e3-b205-51d6184c5778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:03:06.189930 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:03:06.203931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c3da9-f096-47e3-b205-51d6184c5778 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:03:06.207441 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b011f0a4-4fdb-4cf2-a8d5-3dc8ac7132b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:03:06.208155 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011f0a4-4fdb-4cf2-a8d5-3dc8ac7132b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:03:06.208433 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011f0a4-4fdb-4cf2-a8d5-3dc8ac7132b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:03:06.208701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011f0a4-4fdb-4cf2-a8d5-3dc8ac7132b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:03:06.218481 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b011f0a4-4fdb-4cf2-a8d5-3dc8ac7132b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:03:06.219970 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b011f0a4-4fdb-4cf2-a8d5-3dc8ac7132b8 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:03:06.229118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 775/1536] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:03:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:03:08.386144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9def61-d3b2-47c9-8ab6-b13d85814e3d None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:03:08.386809 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9def61-d3b2-47c9-8ab6-b13d85814e3d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:03:08.387002 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9def61-d3b2-47c9-8ab6-b13d85814e3d None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:03:08.388424 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 762/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 01:03:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:03:08.393953 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c9608-b32b-4b50-ac0f-0e4e238eb07e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:03:08.394409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c9608-b32b-4b50-ac0f-0e4e238eb07e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:03:08.394520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c9608-b32b-4b50-ac0f-0e4e238eb07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:03:08.414428 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-472c9608-b32b-4b50-ac0f-0e4e238eb07e None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:03:08.415651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-472c9608-b32b-4b50-ac0f-0e4e238eb07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:03:08.423831 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 776/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 01:03:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:03:08.732447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9def61-d3b2-47c9-8ab6-b13d85814e3d None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:03:08.734088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-aa9def61-d3b2-47c9-8ab6-b13d85814e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:03:08.747723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9def61-d3b2-47c9-8ab6-b13d85814e3d None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:03:08.751270 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22e5b0-2f46-422b-9346-3885d711d736 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:03:08.751912 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22e5b0-2f46-422b-9346-3885d711d736 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:03:08.752192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22e5b0-2f46-422b-9346-3885d711d736 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:03:08.752465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22e5b0-2f46-422b-9346-3885d711d736 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:03:08.761816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d22e5b0-2f46-422b-9346-3885d711d736 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:03:08.763240 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d22e5b0-2f46-422b-9346-3885d711d736 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:03:08.773007 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 763/1539] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:03:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:03:18.409586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=11 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:03:18.768707 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=8 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:35.218429 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95aa46-39e7-4ae9-a6c6-0ef6caac427c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.220080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95aa46-39e7-4ae9-a6c6-0ef6caac427c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.220080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95aa46-39e7-4ae9-a6c6-0ef6caac427c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:35.250676 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7f95aa46-39e7-4ae9-a6c6-0ef6caac427c None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:35.252110 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7f95aa46-39e7-4ae9-a6c6-0ef6caac427c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:35.265914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 777/1540] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:35.274170 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e76233-e56c-41b2-b256-db71f698a91e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.275207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e76233-e56c-41b2-b256-db71f698a91e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.275331 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2f2eb-2b77-46d4-aaa4-a19bfdfb4b20 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.275521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2f2eb-2b77-46d4-aaa4-a19bfdfb4b20 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.275826 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2f2eb-2b77-46d4-aaa4-a19bfdfb4b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:35.276279 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e76233-e56c-41b2-b256-db71f698a91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:35.301412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-aab2f2eb-2b77-46d4-aaa4-a19bfdfb4b20 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:35.302784 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-aab2f2eb-2b77-46d4-aaa4-a19bfdfb4b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:35.306937 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d9e76233-e56c-41b2-b256-db71f698a91e None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:35.308238 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d9e76233-e56c-41b2-b256-db71f698a91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:35.320869 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 778/1541] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:04:35.325416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2f2eb-2b77-46d4-aaa4-a19bfdfb4b20 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:35.328720 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 764/1542] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:35.338221 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff73344-0cde-43d4-84ac-dc9e8d9a9e22 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.338506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff73344-0cde-43d4-84ac-dc9e8d9a9e22 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.338989 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff73344-0cde-43d4-84ac-dc9e8d9a9e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:35.341866 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2f2eb-2b77-46d4-aaa4-a19bfdfb4b20 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:35.349362 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29490b00-7bc4-4f6f-aa99-36ab711495a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:35.349997 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29490b00-7bc4-4f6f-aa99-36ab711495a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.350275 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29490b00-7bc4-4f6f-aa99-36ab711495a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.350546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29490b00-7bc4-4f6f-aa99-36ab711495a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:35.351826 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29490b00-7bc4-4f6f-aa99-36ab711495a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:04:35.352169 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29490b00-7bc4-4f6f-aa99-36ab711495a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:35.354204 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29490b00-7bc4-4f6f-aa99-36ab711495a2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:35.375338 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 779/1543] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:04:35.376851 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19685b20-91cc-427e-b28c-7528d5e4c770 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.377150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19685b20-91cc-427e-b28c-7528d5e4c770 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.377424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19685b20-91cc-427e-b28c-7528d5e4c770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:35.378954 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-3ff73344-0cde-43d4-84ac-dc9e8d9a9e22 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:35.380376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-3ff73344-0cde-43d4-84ac-dc9e8d9a9e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:35.402250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 765/1544] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:04:35.403398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff73344-0cde-43d4-84ac-dc9e8d9a9e22 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:35.406508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-19685b20-91cc-427e-b28c-7528d5e4c770 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:35.407869 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-19685b20-91cc-427e-b28c-7528d5e4c770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:35.420208 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff73344-0cde-43d4-84ac-dc9e8d9a9e22 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:35.424318 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 780/1545] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:35.425142 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19685b20-91cc-427e-b28c-7528d5e4c770 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:35.430624 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698387f-4c14-4116-a521-04ec82210147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:35.431709 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698387f-4c14-4116-a521-04ec82210147 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.432167 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698387f-4c14-4116-a521-04ec82210147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.432613 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698387f-4c14-4116-a521-04ec82210147 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:35.433423 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698387f-4c14-4116-a521-04ec82210147 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:35.435507 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698387f-4c14-4116-a521-04ec82210147 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:35.442453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19685b20-91cc-427e-b28c-7528d5e4c770 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:35.457418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca511a80-f602-4c29-a776-e0eaef8a391f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:35.457418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca511a80-f602-4c29-a776-e0eaef8a391f admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.457418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca511a80-f602-4c29-a776-e0eaef8a391f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.457418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca511a80-f602-4c29-a776-e0eaef8a391f admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:35.457929 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca511a80-f602-4c29-a776-e0eaef8a391f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:04:35.457929 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca511a80-f602-4c29-a776-e0eaef8a391f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:35.457929 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca511a80-f602-4c29-a776-e0eaef8a391f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:35.483312 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 781/1546] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:04:35.484564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd812ac1-653b-4a93-aa8c-3fa2866f90fd None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.484853 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd812ac1-653b-4a93-aa8c-3fa2866f90fd None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.485130 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd812ac1-653b-4a93-aa8c-3fa2866f90fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:35.517032 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 766/1547] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:04:35 2022] POST /identity/v3/projects => generated 393 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:35.525394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-dd812ac1-653b-4a93-aa8c-3fa2866f90fd None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:35.525584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f753c41-2eba-491e-9239-83ee17489f92 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.525697 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f753c41-2eba-491e-9239-83ee17489f92 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.525959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f753c41-2eba-491e-9239-83ee17489f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:35.526715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dd812ac1-653b-4a93-aa8c-3fa2866f90fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:35.556537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-4f753c41-2eba-491e-9239-83ee17489f92 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:35.557836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4f753c41-2eba-491e-9239-83ee17489f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:35.633642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 782/1548] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:35.633874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 767/1549] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:35.633874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f753c41-2eba-491e-9239-83ee17489f92 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:35.634054 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd812ac1-653b-4a93-aa8c-3fa2866f90fd None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:35.641174 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f753c41-2eba-491e-9239-83ee17489f92 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:35.646115 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f3cc0-d64f-4700-bcfb-da06d728373f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:35.646720 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f3cc0-d64f-4700-bcfb-da06d728373f admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.647000 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f3cc0-d64f-4700-bcfb-da06d728373f admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.647272 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f3cc0-d64f-4700-bcfb-da06d728373f admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:35.647944 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6f3cc0-d64f-4700-bcfb-da06d728373f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:35.649415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: WARNING py.warnings [None req-3e6f3cc0-d64f-4700-bcfb-da06d728373f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 01:04:35.649415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: warnings.warn(msg) Sep 14 01:04:35.649415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: Sep 14 01:04:35.650803 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd812ac1-653b-4a93-aa8c-3fa2866f90fd None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:35.650952 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6f3cc0-d64f-4700-bcfb-da06d728373f admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:35.656003 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4219c11-aa23-485f-a4d8-d63376f99dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:35.656644 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4219c11-aa23-485f-a4d8-d63376f99dda admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.656927 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4219c11-aa23-485f-a4d8-d63376f99dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.657200 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4219c11-aa23-485f-a4d8-d63376f99dda admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:35.659632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4219c11-aa23-485f-a4d8-d63376f99dda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:35.659632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: WARNING py.warnings [None req-c4219c11-aa23-485f-a4d8-d63376f99dda admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 01:04:35.659632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: warnings.warn(msg) Sep 14 01:04:35.659632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: Sep 14 01:04:35.660847 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4219c11-aa23-485f-a4d8-d63376f99dda admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:35.673340 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 768/1550] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:04:35 2022] POST /identity/v3/users => generated 297 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:35.674579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ad380-be92-49fd-a784-7ea52026f9d1 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.674930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ad380-be92-49fd-a784-7ea52026f9d1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.675212 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ad380-be92-49fd-a784-7ea52026f9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:35.697378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 783/1551] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:04:35 2022] POST /identity/v3/projects => generated 377 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:35.697378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d952e-ee4a-487a-8a01-6e04f0ef1bc7 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.697378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d952e-ee4a-487a-8a01-6e04f0ef1bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.697378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d952e-ee4a-487a-8a01-6e04f0ef1bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:35.697862 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-f66ad380-be92-49fd-a784-7ea52026f9d1 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:35.697981 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f66ad380-be92-49fd-a784-7ea52026f9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:35.715291 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e74d952e-ee4a-487a-8a01-6e04f0ef1bc7 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:35.716534 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e74d952e-ee4a-487a-8a01-6e04f0ef1bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:35.759051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 769/1552] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:35.760448 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5b042-ca53-455b-b67a-9040b2bf90f7 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.760738 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5b042-ca53-455b-b67a-9040b2bf90f7 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.761011 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5b042-ca53-455b-b67a-9040b2bf90f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:35.782807 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-73e5b042-ca53-455b-b67a-9040b2bf90f7 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:35.784138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-73e5b042-ca53-455b-b67a-9040b2bf90f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:35.787048 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 784/1553] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:35.787924 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d952e-ee4a-487a-8a01-6e04f0ef1bc7 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:35.797376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 770/1554] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:35.798887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a53b4-e151-48fa-94e0-6ca81b92700b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.799175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a53b4-e151-48fa-94e0-6ca81b92700b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.799348 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d952e-ee4a-487a-8a01-6e04f0ef1bc7 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:35.799599 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a53b4-e151-48fa-94e0-6ca81b92700b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:35.804176 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717816b-b84a-4e3d-befa-dede9a56f7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:35.805333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717816b-b84a-4e3d-befa-dede9a56f7b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.805605 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717816b-b84a-4e3d-befa-dede9a56f7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.805932 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717816b-b84a-4e3d-befa-dede9a56f7b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:35.806592 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3717816b-b84a-4e3d-befa-dede9a56f7b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:04:35.806898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3717816b-b84a-4e3d-befa-dede9a56f7b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:35.808538 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3717816b-b84a-4e3d-befa-dede9a56f7b5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:35.821582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 785/1555] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:04:35.823179 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-855a53b4-e151-48fa-94e0-6ca81b92700b None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:35.824159 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4c3c7-1fb9-4d1b-ac77-fd32237b8181 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.824709 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-855a53b4-e151-48fa-94e0-6ca81b92700b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:35.824825 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4c3c7-1fb9-4d1b-ac77-fd32237b8181 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.825301 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4c3c7-1fb9-4d1b-ac77-fd32237b8181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:35.833306 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 771/1556] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:35.834122 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a53b4-e151-48fa-94e0-6ca81b92700b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:35.843660 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a53b4-e151-48fa-94e0-6ca81b92700b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:35.848250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd37b9-5f1f-465c-84df-8a2049582049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:35.848851 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd37b9-5f1f-465c-84df-8a2049582049 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.849132 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd37b9-5f1f-465c-84df-8a2049582049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.849415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd37b9-5f1f-465c-84df-8a2049582049 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:35.850057 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd37b9-5f1f-465c-84df-8a2049582049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:35.850411 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-67c4c3c7-1fb9-4d1b-ac77-fd32237b8181 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:35.851708 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd37b9-5f1f-465c-84df-8a2049582049 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:35.851982 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-67c4c3c7-1fb9-4d1b-ac77-fd32237b8181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:35.861498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 786/1557] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:35.862163 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 772/1558] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:35.862311 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4c3c7-1fb9-4d1b-ac77-fd32237b8181 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:35.863210 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645d6e6-f651-4cca-b338-e01c4b8ac1a4 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.863502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645d6e6-f651-4cca-b338-e01c4b8ac1a4 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.863801 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645d6e6-f651-4cca-b338-e01c4b8ac1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:35.876831 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4c3c7-1fb9-4d1b-ac77-fd32237b8181 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:35.881468 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a0f65-b925-4562-b57f-6bbd5daefd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:35.882067 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a0f65-b925-4562-b57f-6bbd5daefd8a admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.882343 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a0f65-b925-4562-b57f-6bbd5daefd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.882688 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a0f65-b925-4562-b57f-6bbd5daefd8a admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:35.883742 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a0f65-b925-4562-b57f-6bbd5daefd8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:35.884544 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a645d6e6-f651-4cca-b338-e01c4b8ac1a4 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:35.885655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a0f65-b925-4562-b57f-6bbd5daefd8a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:35.885763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a645d6e6-f651-4cca-b338-e01c4b8ac1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:35.894040 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 773/1559] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:35.895106 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe684bda-7916-43ee-b081-956a2af3e63e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.895391 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe684bda-7916-43ee-b081-956a2af3e63e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.895697 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe684bda-7916-43ee-b081-956a2af3e63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:35.898043 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:35.911789 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 787/1560] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:04:35 2022] POST /identity/v3/users => generated 289 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:35.913088 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5160df-2077-4db4-a9a5-221a76b33b67 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.913480 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5160df-2077-4db4-a9a5-221a76b33b67 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.914389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5160df-2077-4db4-a9a5-221a76b33b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:35.922753 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-fe684bda-7916-43ee-b081-956a2af3e63e None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:35.923977 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-fe684bda-7916-43ee-b081-956a2af3e63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:35.932433 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 774/1561] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:35.933229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe684bda-7916-43ee-b081-956a2af3e63e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:35.934090 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-5b5160df-2077-4db4-a9a5-221a76b33b67 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:35.935292 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5b5160df-2077-4db4-a9a5-221a76b33b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:35.937958 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:35.944066 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 788/1562] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:35.944522 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe684bda-7916-43ee-b081-956a2af3e63e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:35.944839 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5160df-2077-4db4-a9a5-221a76b33b67 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:35.950730 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa897df-d022-4be0-bd70-5ff8e4f9ef55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:35.951359 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa897df-d022-4be0-bd70-5ff8e4f9ef55 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.951652 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa897df-d022-4be0-bd70-5ff8e4f9ef55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.951929 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa897df-d022-4be0-bd70-5ff8e4f9ef55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:35.952561 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa897df-d022-4be0-bd70-5ff8e4f9ef55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:35.954465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa897df-d022-4be0-bd70-5ff8e4f9ef55 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:35.954630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5160df-2077-4db4-a9a5-221a76b33b67 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:35.959279 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55254175-3916-4d7c-84e2-c599c359a3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:35.959933 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55254175-3916-4d7c-84e2-c599c359a3a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.960218 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55254175-3916-4d7c-84e2-c599c359a3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.960491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55254175-3916-4d7c-84e2-c599c359a3a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:35.961848 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55254175-3916-4d7c-84e2-c599c359a3a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:04:35.962147 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55254175-3916-4d7c-84e2-c599c359a3a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:35.963776 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55254175-3916-4d7c-84e2-c599c359a3a8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:35.975942 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 789/1563] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:04:35.977950 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55254175-3916-4d7c-84e2-c599c359a3a8 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:35.990543 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 775/1564] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:04:35 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:35.990543 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7a1be-8f08-4131-be66-055dc781e9f1 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:35.990813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55254175-3916-4d7c-84e2-c599c359a3a8 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:35.990919 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7a1be-8f08-4131-be66-055dc781e9f1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:35.990919 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7a1be-8f08-4131-be66-055dc781e9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.008374 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-acf7a1be-8f08-4131-be66-055dc781e9f1 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:36.009588 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-acf7a1be-8f08-4131-be66-055dc781e9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:36.048279 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2f489-a0d5-4bd2-b8ad-fec7cc0f39e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.048890 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2f489-a0d5-4bd2-b8ad-fec7cc0f39e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.049175 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2f489-a0d5-4bd2-b8ad-fec7cc0f39e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.049473 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2f489-a0d5-4bd2-b8ad-fec7cc0f39e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.050120 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea2f489-a0d5-4bd2-b8ad-fec7cc0f39e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.051877 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea2f489-a0d5-4bd2-b8ad-fec7cc0f39e7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.061725 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 790/1565] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:36.062548 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2f489-a0d5-4bd2-b8ad-fec7cc0f39e7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.072320 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 776/1566] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:36.073179 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf7a1be-8f08-4131-be66-055dc781e9f1 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.075131 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2f489-a0d5-4bd2-b8ad-fec7cc0f39e7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.079989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e6e45-1ed9-4f19-adea-1b19e1c6b86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.080639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e6e45-1ed9-4f19-adea-1b19e1c6b86f admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.080925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e6e45-1ed9-4f19-adea-1b19e1c6b86f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.081201 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e6e45-1ed9-4f19-adea-1b19e1c6b86f admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.081866 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e6e45-1ed9-4f19-adea-1b19e1c6b86f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:04:36.082133 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf7a1be-8f08-4131-be66-055dc781e9f1 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.082234 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e6e45-1ed9-4f19-adea-1b19e1c6b86f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.083954 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e6e45-1ed9-4f19-adea-1b19e1c6b86f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.088238 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366fe28-04e4-412d-9d84-9a702b08473a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.088831 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366fe28-04e4-412d-9d84-9a702b08473a admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.089110 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366fe28-04e4-412d-9d84-9a702b08473a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.089383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7366fe28-04e4-412d-9d84-9a702b08473a admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.090012 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7366fe28-04e4-412d-9d84-9a702b08473a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.091615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7366fe28-04e4-412d-9d84-9a702b08473a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.096999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 791/1567] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:04:36.096999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e6e45-1ed9-4f19-adea-1b19e1c6b86f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.104940 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 777/1568] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:36.105344 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366fe28-04e4-412d-9d84-9a702b08473a admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.112331 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e6e45-1ed9-4f19-adea-1b19e1c6b86f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.114839 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7366fe28-04e4-412d-9d84-9a702b08473a admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.116977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba8c43-d7f6-403f-be73-0bb8bacdc62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.117575 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba8c43-d7f6-403f-be73-0bb8bacdc62e admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.117961 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba8c43-d7f6-403f-be73-0bb8bacdc62e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.118241 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba8c43-d7f6-403f-be73-0bb8bacdc62e admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.118884 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba8c43-d7f6-403f-be73-0bb8bacdc62e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:04:36.119190 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba8c43-d7f6-403f-be73-0bb8bacdc62e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.119770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f918d-326e-4a9f-99ac-2177c524f9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.120361 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f918d-326e-4a9f-99ac-2177c524f9ac admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.120651 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f918d-326e-4a9f-99ac-2177c524f9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.120863 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba8c43-d7f6-403f-be73-0bb8bacdc62e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.120979 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f918d-326e-4a9f-99ac-2177c524f9ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.121569 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f918d-326e-4a9f-99ac-2177c524f9ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:04:36.121868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f918d-326e-4a9f-99ac-2177c524f9ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.122877 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: WARNING py.warnings [None req-1d9f918d-326e-4a9f-99ac-2177c524f9ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 14 01:04:36.122877 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: warnings.warn(msg) Sep 14 01:04:36.122877 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: Sep 14 01:04:36.123790 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f918d-326e-4a9f-99ac-2177c524f9ac admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.133157 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 792/1569] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:04:36.141008 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba8c43-d7f6-403f-be73-0bb8bacdc62e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.141114 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 778/1570] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:04:36.141114 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f918d-326e-4a9f-99ac-2177c524f9ac admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.145862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba8c43-d7f6-403f-be73-0bb8bacdc62e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.150636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78110a9-bda0-4c2d-a4e5-43c94b40e0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.151228 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78110a9-bda0-4c2d-a4e5-43c94b40e0cd admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.151511 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78110a9-bda0-4c2d-a4e5-43c94b40e0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.151812 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78110a9-bda0-4c2d-a4e5-43c94b40e0cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.152475 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78110a9-bda0-4c2d-a4e5-43c94b40e0cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.153441 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f918d-326e-4a9f-99ac-2177c524f9ac admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.154378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78110a9-bda0-4c2d-a4e5-43c94b40e0cd admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.158225 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57f9fb-c65a-46cf-84e8-2a7f150f14d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.158928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57f9fb-c65a-46cf-84e8-2a7f150f14d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.159151 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57f9fb-c65a-46cf-84e8-2a7f150f14d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.159456 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57f9fb-c65a-46cf-84e8-2a7f150f14d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.160114 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57f9fb-c65a-46cf-84e8-2a7f150f14d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.162030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57f9fb-c65a-46cf-84e8-2a7f150f14d0 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.184253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 793/1571] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:04:36 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:36.185101 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78110a9-bda0-4c2d-a4e5-43c94b40e0cd admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.190815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 779/1572] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:04:36 2022] POST /identity/v3/users => generated 287 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:36.190815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57f9fb-c65a-46cf-84e8-2a7f150f14d0 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.200267 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57f9fb-c65a-46cf-84e8-2a7f150f14d0 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.265085 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78110a9-bda0-4c2d-a4e5-43c94b40e0cd admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.268185 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e4c03d-3d06-4d9f-ad1b-876fb29b64ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.268800 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4c03d-3d06-4d9f-ad1b-876fb29b64ce admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.269085 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4c03d-3d06-4d9f-ad1b-876fb29b64ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.269366 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4c03d-3d06-4d9f-ad1b-876fb29b64ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.269924 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b42617-49d9-4fca-8c70-7057f90d101f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.270040 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e4c03d-3d06-4d9f-ad1b-876fb29b64ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:04:36.270306 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e4c03d-3d06-4d9f-ad1b-876fb29b64ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.270585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b42617-49d9-4fca-8c70-7057f90d101f admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.270866 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b42617-49d9-4fca-8c70-7057f90d101f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.271140 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b42617-49d9-4fca-8c70-7057f90d101f admin admin] PATH_INFO: `/v3/projects/d1dd868f069a46dbb6bfd549f6bbf7b8/users/bf73eca572454b88843785d791f0b471/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.271957 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e4c03d-3d06-4d9f-ad1b-876fb29b64ce admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.284344 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 780/1573] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:04:36.285241 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e4c03d-3d06-4d9f-ad1b-876fb29b64ce admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.293494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b42617-49d9-4fca-8c70-7057f90d101f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1dd868f069a46dbb6bfd549f6bbf7b8, user_id=bf73eca572454b88843785d791f0b471, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.294581 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: WARNING py.warnings [None req-06b42617-49d9-4fca-8c70-7057f90d101f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 01:04:36.294581 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: warnings.warn(msg) Sep 14 01:04:36.294581 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: Sep 14 01:04:36.295498 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e4c03d-3d06-4d9f-ad1b-876fb29b64ce admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.296605 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b42617-49d9-4fca-8c70-7057f90d101f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.300273 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59590d-af69-4c4c-addd-6c4dde4c705e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.300907 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59590d-af69-4c4c-addd-6c4dde4c705e admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.301164 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59590d-af69-4c4c-addd-6c4dde4c705e admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.301443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59590d-af69-4c4c-addd-6c4dde4c705e admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.302095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59590d-af69-4c4c-addd-6c4dde4c705e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.304071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59590d-af69-4c4c-addd-6c4dde4c705e admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.311315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 794/1574] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:04:36 2022] PUT /identity/v3/projects/d1dd868f069a46dbb6bfd549f6bbf7b8/users/bf73eca572454b88843785d791f0b471/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:36.312234 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b42617-49d9-4fca-8c70-7057f90d101f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.324186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:36.332885 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 781/1575] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:04:36 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:36.333916 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59590d-af69-4c4c-addd-6c4dde4c705e admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.346296 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:36.380735 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b42617-49d9-4fca-8c70-7057f90d101f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.397550 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59590d-af69-4c4c-addd-6c4dde4c705e admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.402973 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8dd9c-aab8-47b1-8417-b23afe196731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.403608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8dd9c-aab8-47b1-8417-b23afe196731 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.403897 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8dd9c-aab8-47b1-8417-b23afe196731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.404173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8dd9c-aab8-47b1-8417-b23afe196731 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.404806 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e8dd9c-aab8-47b1-8417-b23afe196731 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.406726 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e8dd9c-aab8-47b1-8417-b23afe196731 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.435115 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 782/1576] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:04:36 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:36.435833 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8dd9c-aab8-47b1-8417-b23afe196731 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.441067 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f6205-e7aa-4909-9bf1-54bb6adc27cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.441718 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f6205-e7aa-4909-9bf1-54bb6adc27cb admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.441999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f6205-e7aa-4909-9bf1-54bb6adc27cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.442353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f6205-e7aa-4909-9bf1-54bb6adc27cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.442996 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f6205-e7aa-4909-9bf1-54bb6adc27cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.444642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f6205-e7aa-4909-9bf1-54bb6adc27cb admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.455016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 795/1577] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:36.459212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f6205-e7aa-4909-9bf1-54bb6adc27cb admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.469256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f6205-e7aa-4909-9bf1-54bb6adc27cb admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.503352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8dd9c-aab8-47b1-8417-b23afe196731 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.508174 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952522f1-0508-479d-81e9-de47f6c42b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.508759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952522f1-0508-479d-81e9-de47f6c42b58 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.509062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952522f1-0508-479d-81e9-de47f6c42b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.509317 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952522f1-0508-479d-81e9-de47f6c42b58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.509946 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952522f1-0508-479d-81e9-de47f6c42b58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.511884 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952522f1-0508-479d-81e9-de47f6c42b58 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.532059 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a905f2-f1e0-462c-9abb-bcb0fa8688e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.532617 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a905f2-f1e0-462c-9abb-bcb0fa8688e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.532904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a905f2-f1e0-462c-9abb-bcb0fa8688e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.533182 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a905f2-f1e0-462c-9abb-bcb0fa8688e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.534004 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a905f2-f1e0-462c-9abb-bcb0fa8688e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.535985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a905f2-f1e0-462c-9abb-bcb0fa8688e1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.540691 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 783/1578] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:04:36 2022] POST /identity/v3/projects => generated 389 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:36.541554 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952522f1-0508-479d-81e9-de47f6c42b58 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.558748 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 796/1579] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:04:36 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:36.559607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a905f2-f1e0-462c-9abb-bcb0fa8688e1 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.569271 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a905f2-f1e0-462c-9abb-bcb0fa8688e1 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.606277 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952522f1-0508-479d-81e9-de47f6c42b58 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.606902 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:36.613967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e788bfdf-a618-4ddb-9636-ff44d050d785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.614583 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788bfdf-a618-4ddb-9636-ff44d050d785 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.614862 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788bfdf-a618-4ddb-9636-ff44d050d785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.615144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788bfdf-a618-4ddb-9636-ff44d050d785 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.615819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e788bfdf-a618-4ddb-9636-ff44d050d785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.618052 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e788bfdf-a618-4ddb-9636-ff44d050d785 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.631485 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 784/1580] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:36.631485 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac38a4-5165-497e-b002-f570f335c515 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.631485 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac38a4-5165-497e-b002-f570f335c515 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.631485 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac38a4-5165-497e-b002-f570f335c515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.631938 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0c37c-b42d-4e2d-a4fe-5031b711cd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.632269 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0c37c-b42d-4e2d-a4fe-5031b711cd56 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.633380 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0c37c-b42d-4e2d-a4fe-5031b711cd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.633631 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0c37c-b42d-4e2d-a4fe-5031b711cd56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.634526 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd0c37c-b42d-4e2d-a4fe-5031b711cd56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.636500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd0c37c-b42d-4e2d-a4fe-5031b711cd56 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.650273 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-5fac38a4-5165-497e-b002-f570f335c515 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:36.651495 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5fac38a4-5165-497e-b002-f570f335c515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:36.659439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 785/1581] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:36.660320 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac38a4-5165-497e-b002-f570f335c515 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.663791 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 797/1582] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:04:36 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:36.664805 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0c37c-b42d-4e2d-a4fe-5031b711cd56 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.673612 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac38a4-5165-497e-b002-f570f335c515 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.682211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:36.728737 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0c37c-b42d-4e2d-a4fe-5031b711cd56 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.731880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f661b15-a00c-4394-aeec-667e249b9577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.732498 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f661b15-a00c-4394-aeec-667e249b9577 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.732782 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f661b15-a00c-4394-aeec-667e249b9577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.733061 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f661b15-a00c-4394-aeec-667e249b9577 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.733547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b99735-7a41-43c3-9b23-346d304c1483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.733792 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f661b15-a00c-4394-aeec-667e249b9577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.734217 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b99735-7a41-43c3-9b23-346d304c1483 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.734625 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b99735-7a41-43c3-9b23-346d304c1483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.735106 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b99735-7a41-43c3-9b23-346d304c1483 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.735780 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b99735-7a41-43c3-9b23-346d304c1483 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.735902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f661b15-a00c-4394-aeec-667e249b9577 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.737687 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b99735-7a41-43c3-9b23-346d304c1483 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.759702 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 798/1584] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:04:36 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:36.759916 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 786/1584] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:04:36 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:36.763641 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b99735-7a41-43c3-9b23-346d304c1483 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.763791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f661b15-a00c-4394-aeec-667e249b9577 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.774223 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b99735-7a41-43c3-9b23-346d304c1483 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.774497 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f661b15-a00c-4394-aeec-667e249b9577 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.778818 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7742ca4-04d5-4e82-8266-4f147299bb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.778986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1279a-0ed0-441c-b404-f6aa26afdfff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.779469 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7742ca4-04d5-4e82-8266-4f147299bb43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.779605 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1279a-0ed0-441c-b404-f6aa26afdfff admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.779782 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7742ca4-04d5-4e82-8266-4f147299bb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.779887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1279a-0ed0-441c-b404-f6aa26afdfff admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.780090 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7742ca4-04d5-4e82-8266-4f147299bb43 admin admin] PATH_INFO: `/v3/projects/25c229f6a4574619a73b2780d2a6528b/users/2eca12e1e0a64eeeb183a328f338f54d/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.780187 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1279a-0ed0-441c-b404-f6aa26afdfff admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.780709 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1279a-0ed0-441c-b404-f6aa26afdfff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.782685 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1279a-0ed0-441c-b404-f6aa26afdfff admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.795649 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7742ca4-04d5-4e82-8266-4f147299bb43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25c229f6a4574619a73b2780d2a6528b, user_id=2eca12e1e0a64eeeb183a328f338f54d, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.798385 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7742ca4-04d5-4e82-8266-4f147299bb43 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.804070 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 787/1585] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:04:36 2022] POST /identity/v3/users => generated 295 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:36.804070 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1279a-0ed0-441c-b404-f6aa26afdfff admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.813966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 799/1586] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:04:36 2022] PUT /identity/v3/projects/25c229f6a4574619a73b2780d2a6528b/users/2eca12e1e0a64eeeb183a328f338f54d/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:36.814936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7742ca4-04d5-4e82-8266-4f147299bb43 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.815085 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1279a-0ed0-441c-b404-f6aa26afdfff admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.876184 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56d7b63-56f0-4015-9f76-628521ce2c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.876796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d7b63-56f0-4015-9f76-628521ce2c88 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.877075 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d7b63-56f0-4015-9f76-628521ce2c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.877348 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d7b63-56f0-4015-9f76-628521ce2c88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.877981 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56d7b63-56f0-4015-9f76-628521ce2c88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.878481 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7742ca4-04d5-4e82-8266-4f147299bb43 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.879599 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56d7b63-56f0-4015-9f76-628521ce2c88 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.883409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92622a9-cdbc-446d-920b-5dd140db72f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.884050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92622a9-cdbc-446d-920b-5dd140db72f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.884332 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92622a9-cdbc-446d-920b-5dd140db72f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.884613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92622a9-cdbc-446d-920b-5dd140db72f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.885259 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92622a9-cdbc-446d-920b-5dd140db72f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.886882 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92622a9-cdbc-446d-920b-5dd140db72f0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.889932 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 788/1587] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:36.890839 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56d7b63-56f0-4015-9f76-628521ce2c88 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.900579 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 800/1588] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:36.900579 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92622a9-cdbc-446d-920b-5dd140db72f0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.903266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56d7b63-56f0-4015-9f76-628521ce2c88 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.907894 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb198d-b2d0-4b3d-8bcf-a249bb6ddf08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.908488 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb198d-b2d0-4b3d-8bcf-a249bb6ddf08 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.908768 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb198d-b2d0-4b3d-8bcf-a249bb6ddf08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.909074 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb198d-b2d0-4b3d-8bcf-a249bb6ddf08 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.909694 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb198d-b2d0-4b3d-8bcf-a249bb6ddf08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.911711 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb198d-b2d0-4b3d-8bcf-a249bb6ddf08 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.928834 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:36.935103 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 789/1589] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:04:36 2022] POST /identity/v3/users => generated 286 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:36.936061 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb198d-b2d0-4b3d-8bcf-a249bb6ddf08 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.945760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb198d-b2d0-4b3d-8bcf-a249bb6ddf08 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.950569 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe59cc-fa66-48b5-8bbb-d192796ff224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.951164 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe59cc-fa66-48b5-8bbb-d192796ff224 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.951444 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe59cc-fa66-48b5-8bbb-d192796ff224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.951760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe59cc-fa66-48b5-8bbb-d192796ff224 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.952392 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe59cc-fa66-48b5-8bbb-d192796ff224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.954021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe59cc-fa66-48b5-8bbb-d192796ff224 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.960465 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92622a9-cdbc-446d-920b-5dd140db72f0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.964439 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aae704-5d3c-465a-b59d-5e1e802cba91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.964930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 790/1590] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:36.965188 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aae704-5d3c-465a-b59d-5e1e802cba91 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.965529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aae704-5d3c-465a-b59d-5e1e802cba91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.965809 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe59cc-fa66-48b5-8bbb-d192796ff224 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.965917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aae704-5d3c-465a-b59d-5e1e802cba91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.975347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe59cc-fa66-48b5-8bbb-d192796ff224 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.977743 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aae704-5d3c-465a-b59d-5e1e802cba91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.979174 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aae704-5d3c-465a-b59d-5e1e802cba91 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.980561 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa60bbb-7211-42ed-9cd2-adcc4a4c01ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:36.981143 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa60bbb-7211-42ed-9cd2-adcc4a4c01ac admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.981431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa60bbb-7211-42ed-9cd2-adcc4a4c01ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.981708 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa60bbb-7211-42ed-9cd2-adcc4a4c01ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:36.982330 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa60bbb-7211-42ed-9cd2-adcc4a4c01ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:36.983939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa60bbb-7211-42ed-9cd2-adcc4a4c01ac admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:36.988587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 801/1591] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:04:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:04:36.989379 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aae704-5d3c-465a-b59d-5e1e802cba91 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:36.997768 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 791/1592] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:36.999089 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe0a44-76d8-45e1-bbdd-625c21dd3595 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:36.999467 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aae704-5d3c-465a-b59d-5e1e802cba91 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:36.999615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe0a44-76d8-45e1-bbdd-625c21dd3595 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:36.999800 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe0a44-76d8-45e1-bbdd-625c21dd3595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:37.004229 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69fd46-30a7-418f-ad5f-5874a5037779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:37.005067 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69fd46-30a7-418f-ad5f-5874a5037779 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:37.005067 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69fd46-30a7-418f-ad5f-5874a5037779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:37.005360 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69fd46-30a7-418f-ad5f-5874a5037779 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:37.005994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69fd46-30a7-418f-ad5f-5874a5037779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:37.007732 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69fd46-30a7-418f-ad5f-5874a5037779 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:37.019850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 802/1593] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:37.020735 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-8cfe0a44-76d8-45e1-bbdd-625c21dd3595 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:37.022076 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-8cfe0a44-76d8-45e1-bbdd-625c21dd3595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:37.024075 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69fd46-30a7-418f-ad5f-5874a5037779 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:37.030836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 792/1594] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:37.031591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe0a44-76d8-45e1-bbdd-625c21dd3595 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:37.037573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69fd46-30a7-418f-ad5f-5874a5037779 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:37.049111 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97a7a5-4018-4f19-a84f-eae9e142efbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:37.049731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97a7a5-4018-4f19-a84f-eae9e142efbb admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:37.050041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97a7a5-4018-4f19-a84f-eae9e142efbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:37.050269 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97a7a5-4018-4f19-a84f-eae9e142efbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:37.051181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97a7a5-4018-4f19-a84f-eae9e142efbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:37.053130 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97a7a5-4018-4f19-a84f-eae9e142efbb admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:37.058032 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe0a44-76d8-45e1-bbdd-625c21dd3595 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:37.068230 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade9c88-d35b-4c45-ac01-5be842ad4b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:37.068995 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade9c88-d35b-4c45-ac01-5be842ad4b39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:37.069391 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade9c88-d35b-4c45-ac01-5be842ad4b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:37.069710 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade9c88-d35b-4c45-ac01-5be842ad4b39 admin admin] PATH_INFO: `/v3/projects/277d73e83c4047c098c5e5e38c057b7a/users/e478c96e7ba640ebb95edfec9b029c4c/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:37.071725 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 803/1595] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:37.073317 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97a7a5-4018-4f19-a84f-eae9e142efbb admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:37.084148 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97a7a5-4018-4f19-a84f-eae9e142efbb admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:37.088463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade9c88-d35b-4c45-ac01-5be842ad4b39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=277d73e83c4047c098c5e5e38c057b7a, user_id=e478c96e7ba640ebb95edfec9b029c4c, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:37.089539 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79883b55-ee1e-42e6-996b-64ca0d4a7d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:37.090263 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79883b55-ee1e-42e6-996b-64ca0d4a7d9a admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:37.090753 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79883b55-ee1e-42e6-996b-64ca0d4a7d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:37.091133 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79883b55-ee1e-42e6-996b-64ca0d4a7d9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:37.091472 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade9c88-d35b-4c45-ac01-5be842ad4b39 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:37.092000 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79883b55-ee1e-42e6-996b-64ca0d4a7d9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:37.093712 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79883b55-ee1e-42e6-996b-64ca0d4a7d9a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:37.108222 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 804/1596] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:37.108222 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79883b55-ee1e-42e6-996b-64ca0d4a7d9a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:37.108536 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 793/1597] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:04:37 2022] PUT /identity/v3/projects/277d73e83c4047c098c5e5e38c057b7a/users/e478c96e7ba640ebb95edfec9b029c4c/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:37.108536 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade9c88-d35b-4c45-ac01-5be842ad4b39 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:37.120259 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:37.121412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79883b55-ee1e-42e6-996b-64ca0d4a7d9a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:37.186444 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade9c88-d35b-4c45-ac01-5be842ad4b39 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:37.189078 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed31fde-8e52-4ea2-bdf7-38c82ec66a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:37.189696 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed31fde-8e52-4ea2-bdf7-38c82ec66a86 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:37.189978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed31fde-8e52-4ea2-bdf7-38c82ec66a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:37.190252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed31fde-8e52-4ea2-bdf7-38c82ec66a86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:37.190895 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed31fde-8e52-4ea2-bdf7-38c82ec66a86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:37.192329 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a43d3-848a-44e9-b508-3ca342190476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:37.192582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed31fde-8e52-4ea2-bdf7-38c82ec66a86 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:37.193235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a43d3-848a-44e9-b508-3ca342190476 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:37.193705 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a43d3-848a-44e9-b508-3ca342190476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:37.194048 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a43d3-848a-44e9-b508-3ca342190476 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:37.195319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a43d3-848a-44e9-b508-3ca342190476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:37.197730 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a43d3-848a-44e9-b508-3ca342190476 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:37.211229 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 805/1598] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:37.213042 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed31fde-8e52-4ea2-bdf7-38c82ec66a86 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:37.216439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 794/1599] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:37.217384 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a43d3-848a-44e9-b508-3ca342190476 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:37.227685 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed31fde-8e52-4ea2-bdf7-38c82ec66a86 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:37.233678 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95585f0-29b2-4f1e-adcd-d2b6f7751bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:37.234354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95585f0-29b2-4f1e-adcd-d2b6f7751bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:37.234641 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95585f0-29b2-4f1e-adcd-d2b6f7751bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:37.234945 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95585f0-29b2-4f1e-adcd-d2b6f7751bd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:37.236213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95585f0-29b2-4f1e-adcd-d2b6f7751bd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:37.237653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95585f0-29b2-4f1e-adcd-d2b6f7751bd2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:37.250789 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 806/1600] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:37.251406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95585f0-29b2-4f1e-adcd-d2b6f7751bd2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:37.263794 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95585f0-29b2-4f1e-adcd-d2b6f7751bd2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:37.270580 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ba547-7cd4-480c-9e6f-aa656331a723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:37.271587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ba547-7cd4-480c-9e6f-aa656331a723 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:37.271802 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ba547-7cd4-480c-9e6f-aa656331a723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:37.272087 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ba547-7cd4-480c-9e6f-aa656331a723 admin admin] PATH_INFO: `/v3/projects/62cb41eb4b8244abb4aafd718b1207ad/users/51768890f19140b6819ef89f5bf610c8/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:37.282972 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a43d3-848a-44e9-b508-3ca342190476 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:37.287760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368a0cb-095c-4d0c-bf3b-22a03eec5d8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:37.288460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368a0cb-095c-4d0c-bf3b-22a03eec5d8f service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:37.288558 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368a0cb-095c-4d0c-bf3b-22a03eec5d8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:37.289658 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368a0cb-095c-4d0c-bf3b-22a03eec5d8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:37.290752 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ba547-7cd4-480c-9e6f-aa656331a723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62cb41eb4b8244abb4aafd718b1207ad, user_id=51768890f19140b6819ef89f5bf610c8, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:37.293832 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ba547-7cd4-480c-9e6f-aa656331a723 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:37.300207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368a0cb-095c-4d0c-bf3b-22a03eec5d8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:37.301679 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368a0cb-095c-4d0c-bf3b-22a03eec5d8f service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:37.312397 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 795/1601] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:04:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:04:37.313259 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 807/1602] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:04:37 2022] PUT /identity/v3/projects/62cb41eb4b8244abb4aafd718b1207ad/users/51768890f19140b6819ef89f5bf610c8/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:37.313609 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368a0cb-095c-4d0c-bf3b-22a03eec5d8f service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:37.314560 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecfe8f-71cc-46bc-b964-743046b061cf None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:37.314856 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecfe8f-71cc-46bc-b964-743046b061cf None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:37.315132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecfe8f-71cc-46bc-b964-743046b061cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:37.328822 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368a0cb-095c-4d0c-bf3b-22a03eec5d8f service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:37.349569 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-22ecfe8f-71cc-46bc-b964-743046b061cf None None] MFA Rules not processed for user `e478c96e7ba640ebb95edfec9b029c4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:37.350916 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-22ecfe8f-71cc-46bc-b964-743046b061cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:37.405726 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7344b1-3459-4f2b-82cf-2e066baacc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:37.406483 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7344b1-3459-4f2b-82cf-2e066baacc76 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:37.406907 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7344b1-3459-4f2b-82cf-2e066baacc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:37.407343 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7344b1-3459-4f2b-82cf-2e066baacc76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:37.408132 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7344b1-3459-4f2b-82cf-2e066baacc76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:37.409822 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7344b1-3459-4f2b-82cf-2e066baacc76 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:37.412940 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:37.423864 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 796/1603] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:37.424924 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7344b1-3459-4f2b-82cf-2e066baacc76 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:37.437641 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7344b1-3459-4f2b-82cf-2e066baacc76 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:37.444486 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 808/1604] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:37.445292 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b38ed-d059-4095-8b07-73953a5852fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:37.445469 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ecfe8f-71cc-46bc-b964-743046b061cf None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:37.446645 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b38ed-d059-4095-8b07-73953a5852fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:37.447085 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b38ed-d059-4095-8b07-73953a5852fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:37.447513 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b38ed-d059-4095-8b07-73953a5852fd admin admin] PATH_INFO: `/v3/projects/832072fd5b984b5b91d249c5733452b8/users/017a9de44877436fa9531beb756c54c2/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:37.459760 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ecfe8f-71cc-46bc-b964-743046b061cf None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:37.465300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b38ed-d059-4095-8b07-73953a5852fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=832072fd5b984b5b91d249c5733452b8, user_id=017a9de44877436fa9531beb756c54c2, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:37.465996 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013b279-8a2d-4484-ab37-eac489474243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:37.466787 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013b279-8a2d-4484-ab37-eac489474243 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:37.467171 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013b279-8a2d-4484-ab37-eac489474243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:37.467611 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013b279-8a2d-4484-ab37-eac489474243 admin admin] PATH_INFO: `/v3/projects/9a61a5cdab00450abcb069278b29edf3/users/50f5d7492f704f15b82127fa1c724f38/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:37.475750 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b38ed-d059-4095-8b07-73953a5852fd admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:37.490569 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013b279-8a2d-4484-ab37-eac489474243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a61a5cdab00450abcb069278b29edf3, user_id=50f5d7492f704f15b82127fa1c724f38, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:37.494834 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013b279-8a2d-4484-ab37-eac489474243 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:37.497610 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 797/1605] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:04:37 2022] PUT /identity/v3/projects/832072fd5b984b5b91d249c5733452b8/users/017a9de44877436fa9531beb756c54c2/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:37.498299 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b38ed-d059-4095-8b07-73953a5852fd admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:37.511824 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 809/1606] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:04:37 2022] PUT /identity/v3/projects/9a61a5cdab00450abcb069278b29edf3/users/50f5d7492f704f15b82127fa1c724f38/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:37.513491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83740652-8a42-4576-bb85-de45aa98b105 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:37.513781 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83740652-8a42-4576-bb85-de45aa98b105 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:37.514056 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83740652-8a42-4576-bb85-de45aa98b105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:37.523277 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:37.537037 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-83740652-8a42-4576-bb85-de45aa98b105 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:37.538427 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-83740652-8a42-4576-bb85-de45aa98b105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:37.578311 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b38ed-d059-4095-8b07-73953a5852fd admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:37.606737 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 810/1607] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:37.610666 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83740652-8a42-4576-bb85-de45aa98b105 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:37.621465 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83740652-8a42-4576-bb85-de45aa98b105 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:37.628655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916d3bc-a421-4643-bbce-b5914094a1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:37.629510 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916d3bc-a421-4643-bbce-b5914094a1e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:37.629949 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916d3bc-a421-4643-bbce-b5914094a1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:37.630304 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916d3bc-a421-4643-bbce-b5914094a1e5 admin admin] PATH_INFO: `/v3/projects/437899e27b144fcea3b6c06acc7df525/users/0ded01b9a7c44e89a6ec9d0f8d0bd1a1/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:37.648175 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916d3bc-a421-4643-bbce-b5914094a1e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=437899e27b144fcea3b6c06acc7df525, user_id=0ded01b9a7c44e89a6ec9d0f8d0bd1a1, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:37.651910 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916d3bc-a421-4643-bbce-b5914094a1e5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:37.657524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4a365-6dbe-4c81-97e6-8c35f16f19fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:37.658267 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4a365-6dbe-4c81-97e6-8c35f16f19fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:37.658742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4a365-6dbe-4c81-97e6-8c35f16f19fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:37.659061 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4a365-6dbe-4c81-97e6-8c35f16f19fc admin admin] PATH_INFO: `/v3/projects/3c6482bb91c24939ba18c03a3e5bb18c/users/e4ad2d06f62246be8eb162257cbb4f8e/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:37.669683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 811/1608] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:04:37 2022] PUT /identity/v3/projects/437899e27b144fcea3b6c06acc7df525/users/0ded01b9a7c44e89a6ec9d0f8d0bd1a1/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:37.671183 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916d3bc-a421-4643-bbce-b5914094a1e5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:37.677278 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d4a365-6dbe-4c81-97e6-8c35f16f19fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c6482bb91c24939ba18c03a3e5bb18c, user_id=e4ad2d06f62246be8eb162257cbb4f8e, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:37.680607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d4a365-6dbe-4c81-97e6-8c35f16f19fc admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:37.697123 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 798/1609] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:04:37 2022] PUT /identity/v3/projects/3c6482bb91c24939ba18c03a3e5bb18c/users/e4ad2d06f62246be8eb162257cbb4f8e/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:37.701353 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab78ff-5c39-423f-acda-83fe27d282b2 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:37.701644 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab78ff-5c39-423f-acda-83fe27d282b2 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:37.701925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ab78ff-5c39-423f-acda-83fe27d282b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:37.725765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-05ab78ff-5c39-423f-acda-83fe27d282b2 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:37.727442 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-05ab78ff-5c39-423f-acda-83fe27d282b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:37.754463 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916d3bc-a421-4643-bbce-b5914094a1e5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:37.796281 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 799/1610] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:37.797424 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df25f20-2ae4-4c87-8d1e-96d1fdd5110a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:37.797807 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df25f20-2ae4-4c87-8d1e-96d1fdd5110a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:37.798175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df25f20-2ae4-4c87-8d1e-96d1fdd5110a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:37.822512 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:37.828657 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-7df25f20-2ae4-4c87-8d1e-96d1fdd5110a None None] MFA Rules not processed for user `50f5d7492f704f15b82127fa1c724f38`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:37.829861 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7df25f20-2ae4-4c87-8d1e-96d1fdd5110a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:37.843515 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d7324-6350-48b9-b12d-f715025682bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:37.844431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d7324-6350-48b9-b12d-f715025682bd admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:37.844711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d7324-6350-48b9-b12d-f715025682bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:37.845174 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d7324-6350-48b9-b12d-f715025682bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:37.845662 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8d7324-6350-48b9-b12d-f715025682bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:37.847582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8d7324-6350-48b9-b12d-f715025682bd admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:37.883506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:37.884750 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 812/1611] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:04:37 2022] POST /identity/v3/projects => generated 373 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:37.885670 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d7324-6350-48b9-b12d-f715025682bd admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:37.914436 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 800/1612] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:37.916597 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c140acf-4570-436e-9e45-879ca78d32d7 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:37.917676 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c140acf-4570-436e-9e45-879ca78d32d7 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:37.917676 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c140acf-4570-436e-9e45-879ca78d32d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:37.960348 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-9c140acf-4570-436e-9e45-879ca78d32d7 None None] MFA Rules not processed for user `0ded01b9a7c44e89a6ec9d0f8d0bd1a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:37.962760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9c140acf-4570-436e-9e45-879ca78d32d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:37.968633 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d7324-6350-48b9-b12d-f715025682bd admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:37.974957 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e091e2d-8f8d-4855-9358-a12912ed45a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:37.975888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e091e2d-8f8d-4855-9358-a12912ed45a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:37.976794 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e091e2d-8f8d-4855-9358-a12912ed45a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:37.977140 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e091e2d-8f8d-4855-9358-a12912ed45a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:38.049396 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e091e2d-8f8d-4855-9358-a12912ed45a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:38.050886 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e091e2d-8f8d-4855-9358-a12912ed45a7 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:38.055019 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 801/1613] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:38.056699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09981b27-0a55-4611-a9d2-8fc12c42f6c3 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:38.056989 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09981b27-0a55-4611-a9d2-8fc12c42f6c3 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:38.057300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09981b27-0a55-4611-a9d2-8fc12c42f6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:38.079228 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-09981b27-0a55-4611-a9d2-8fc12c42f6c3 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:38.080814 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-09981b27-0a55-4611-a9d2-8fc12c42f6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:38.095026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 802/1615] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:38.095026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09981b27-0a55-4611-a9d2-8fc12c42f6c3 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:38.095268 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 813/1615] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:04:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:04:38.096836 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e091e2d-8f8d-4855-9358-a12912ed45a7 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:38.122598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e091e2d-8f8d-4855-9358-a12912ed45a7 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:38.124653 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09981b27-0a55-4611-a9d2-8fc12c42f6c3 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:38.130586 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ba397-ab66-48fc-8bd7-749cdf99692e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:38.131112 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ba397-ab66-48fc-8bd7-749cdf99692e service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:38.131976 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ba397-ab66-48fc-8bd7-749cdf99692e service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:38.133674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ba397-ab66-48fc-8bd7-749cdf99692e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:38.134519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4dd47f-1fe1-4c98-8733-d7ebae8fc359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:38.135224 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4dd47f-1fe1-4c98-8733-d7ebae8fc359 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:38.135641 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4dd47f-1fe1-4c98-8733-d7ebae8fc359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:38.135990 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4dd47f-1fe1-4c98-8733-d7ebae8fc359 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:38.141750 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4dd47f-1fe1-4c98-8733-d7ebae8fc359 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:38.146796 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4dd47f-1fe1-4c98-8733-d7ebae8fc359 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:38.149101 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ba397-ab66-48fc-8bd7-749cdf99692e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:38.150726 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ba397-ab66-48fc-8bd7-749cdf99692e service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:38.163431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 803/1616] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:04:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:04:38.164876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07576924-7152-4529-bc10-d6735da7bfc7 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:38.165361 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07576924-7152-4529-bc10-d6735da7bfc7 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:38.165764 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07576924-7152-4529-bc10-d6735da7bfc7 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:38.167805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 804/1617] 10.222.0.42 () {54 vars in 962 bytes} [Wed Sep 14 01:04:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:04:38.169084 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d6d40-b2d8-4dd1-adf6-092a5f5c88ec None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:38.169556 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d6d40-b2d8-4dd1-adf6-092a5f5c88ec None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:38.169934 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d6d40-b2d8-4dd1-adf6-092a5f5c88ec None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:38.171348 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 805/1618] 10.222.0.42 () {54 vars in 962 bytes} [Wed Sep 14 01:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:04:38.172328 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d6d40-b2d8-4dd1-adf6-092a5f5c88ec None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:38.187498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 814/1619] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:04:38 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:38.189315 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d6d40-b2d8-4dd1-adf6-092a5f5c88ec None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:38.189632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7188ce6-c30e-475f-afe9-6f208ed060c5 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:38.189807 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7188ce6-c30e-475f-afe9-6f208ed060c5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:38.190310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7188ce6-c30e-475f-afe9-6f208ed060c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:38.196896 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6973da-f1a5-4a9f-a1a4-a5b320e78f32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:38.197669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6973da-f1a5-4a9f-a1a4-a5b320e78f32 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:38.198022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6973da-f1a5-4a9f-a1a4-a5b320e78f32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:38.198376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6973da-f1a5-4a9f-a1a4-a5b320e78f32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:38.209994 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6973da-f1a5-4a9f-a1a4-a5b320e78f32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:38.212922 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6973da-f1a5-4a9f-a1a4-a5b320e78f32 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:38.231840 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 806/1620] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:04:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:04:38.233567 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a25c6b-a595-48a6-b210-21dbfbd15c98 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:38.233919 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a25c6b-a595-48a6-b210-21dbfbd15c98 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:38.234205 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a25c6b-a595-48a6-b210-21dbfbd15c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:38.275828 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e7188ce6-c30e-475f-afe9-6f208ed060c5 None None] MFA Rules not processed for user `5e0fe2e84f6748c992c9beedf2f5d49b`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:38.279883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e7188ce6-c30e-475f-afe9-6f208ed060c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:38.284864 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e0a25c6b-a595-48a6-b210-21dbfbd15c98 None None] MFA Rules not processed for user `5e0fe2e84f6748c992c9beedf2f5d49b`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:38.287538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e0a25c6b-a595-48a6-b210-21dbfbd15c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:38.409768 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:38.425387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 807/1621] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Sep 14 01:04:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:38.426887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a25c6b-a595-48a6-b210-21dbfbd15c98 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:38.442180 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a25c6b-a595-48a6-b210-21dbfbd15c98 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:38.449173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d2bc7-949d-40c0-bdfa-04a06029dd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:38.449811 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d2bc7-949d-40c0-bdfa-04a06029dd4c admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:38.450180 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d2bc7-949d-40c0-bdfa-04a06029dd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:38.450462 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d2bc7-949d-40c0-bdfa-04a06029dd4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:38.451297 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d2bc7-949d-40c0-bdfa-04a06029dd4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:38.453444 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d2bc7-949d-40c0-bdfa-04a06029dd4c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:38.472527 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 808/1622] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:38 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:38.473469 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d2bc7-949d-40c0-bdfa-04a06029dd4c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:38.484263 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 815/1623] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Sep 14 01:04:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:38.485708 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7188ce6-c30e-475f-afe9-6f208ed060c5 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:38.498917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d2bc7-949d-40c0-bdfa-04a06029dd4c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:38.500644 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7188ce6-c30e-475f-afe9-6f208ed060c5 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:38.507307 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90abebb8-46b1-4d0a-ae8f-97a5a8f3b13a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:38.508687 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abebb8-46b1-4d0a-ae8f-97a5a8f3b13a service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:38.509404 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abebb8-46b1-4d0a-ae8f-97a5a8f3b13a service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:38.512111 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abebb8-46b1-4d0a-ae8f-97a5a8f3b13a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:38.516343 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb7d80-5dca-4ccf-8226-d0c185207060 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:38.518567 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb7d80-5dca-4ccf-8226-d0c185207060 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:38.518857 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb7d80-5dca-4ccf-8226-d0c185207060 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:38.519133 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb7d80-5dca-4ccf-8226-d0c185207060 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:38.545236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb7d80-5dca-4ccf-8226-d0c185207060 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:38.546872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb7d80-5dca-4ccf-8226-d0c185207060 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:38.551515 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:38.566415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 809/1624] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:04:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:04:38.567270 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb7d80-5dca-4ccf-8226-d0c185207060 service nova] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:38.585860 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb7d80-5dca-4ccf-8226-d0c185207060 service nova] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:38.596224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b64a7c1-cc31-4408-92d0-5f52739c05d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:38.597456 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b64a7c1-cc31-4408-92d0-5f52739c05d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:38.597757 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b64a7c1-cc31-4408-92d0-5f52739c05d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:38.598041 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b64a7c1-cc31-4408-92d0-5f52739c05d9 admin admin] PATH_INFO: `/v3/projects/3a1767d9b3cd495bb8e1e9fe4ecb0946/users/1404f53b904946d4981f35880b535d70/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:38.615418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90abebb8-46b1-4d0a-ae8f-97a5a8f3b13a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:38.618061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90abebb8-46b1-4d0a-ae8f-97a5a8f3b13a service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:38.636240 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b64a7c1-cc31-4408-92d0-5f52739c05d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a1767d9b3cd495bb8e1e9fe4ecb0946, user_id=1404f53b904946d4981f35880b535d70, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:38.639230 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b64a7c1-cc31-4408-92d0-5f52739c05d9 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:38.653558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 816/1625] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:04:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:04:38.667039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 810/1626] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:04:38 2022] PUT /identity/v3/projects/3a1767d9b3cd495bb8e1e9fe4ecb0946/users/1404f53b904946d4981f35880b535d70/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:38.671103 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90abebb8-46b1-4d0a-ae8f-97a5a8f3b13a service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:38.684514 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90abebb8-46b1-4d0a-ae8f-97a5a8f3b13a service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:38.780302 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4993fa8-83cf-4a33-89be-02b86589939a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:38.781384 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4993fa8-83cf-4a33-89be-02b86589939a admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:38.781881 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4993fa8-83cf-4a33-89be-02b86589939a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:38.782428 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4993fa8-83cf-4a33-89be-02b86589939a admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:38.783358 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4993fa8-83cf-4a33-89be-02b86589939a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:38.785535 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4993fa8-83cf-4a33-89be-02b86589939a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:38.798691 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 817/1627] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:38 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:04:38.803669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b64a7c1-cc31-4408-92d0-5f52739c05d9 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:38.820596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b64a7c1-cc31-4408-92d0-5f52739c05d9 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:38.840923 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ca0f4-2489-4de1-961f-8e851b78c167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:38.842710 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ca0f4-2489-4de1-961f-8e851b78c167 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:38.843080 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ca0f4-2489-4de1-961f-8e851b78c167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:38.844822 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ca0f4-2489-4de1-961f-8e851b78c167 admin admin] PATH_INFO: `/v3/projects/3a1767d9b3cd495bb8e1e9fe4ecb0946/users/1404f53b904946d4981f35880b535d70/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:38.849858 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ca0f4-2489-4de1-961f-8e851b78c167 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a1767d9b3cd495bb8e1e9fe4ecb0946, user_id=1404f53b904946d4981f35880b535d70, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:38.857986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ca0f4-2489-4de1-961f-8e851b78c167 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:38.900963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 811/1628] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:04:38 2022] PUT /identity/v3/projects/3a1767d9b3cd495bb8e1e9fe4ecb0946/users/1404f53b904946d4981f35880b535d70/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:38.901902 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd4dc8-3f94-46a7-a7c0-76b8e82f241b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:38.902371 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd4dc8-3f94-46a7-a7c0-76b8e82f241b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:38.902722 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd4dc8-3f94-46a7-a7c0-76b8e82f241b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:38.971397 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-42cd4dc8-3f94-46a7-a7c0-76b8e82f241b None None] MFA Rules not processed for user `1404f53b904946d4981f35880b535d70`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:38.974571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-42cd4dc8-3f94-46a7-a7c0-76b8e82f241b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:39.024480 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34659ed1-2567-499a-9ee2-4e79c016b828 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:39.025193 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34659ed1-2567-499a-9ee2-4e79c016b828 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:39.025609 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34659ed1-2567-499a-9ee2-4e79c016b828 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:39.030019 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 812/1629] 10.222.0.42 () {54 vars in 912 bytes} [Wed Sep 14 01:04:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:04:39.043294 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88b9f0-f13b-4df4-a095-2b2efbf2a209 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:39.043600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88b9f0-f13b-4df4-a095-2b2efbf2a209 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:39.043950 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88b9f0-f13b-4df4-a095-2b2efbf2a209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:39.084193 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-4f88b9f0-f13b-4df4-a095-2b2efbf2a209 None None] MFA Rules not processed for user `5e0fe2e84f6748c992c9beedf2f5d49b`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:39.085850 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4f88b9f0-f13b-4df4-a095-2b2efbf2a209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:39.103590 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 818/1630] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:39.116159 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd4dc8-3f94-46a7-a7c0-76b8e82f241b None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:39.121270 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:39.242755 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 813/1631] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 01:04:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:04:39.255833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd4dc8-3f94-46a7-a7c0-76b8e82f241b None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:39.256634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92917221-cc42-43f9-b6f7-10ac130a2dd8 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:39.256931 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92917221-cc42-43f9-b6f7-10ac130a2dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:39.257209 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92917221-cc42-43f9-b6f7-10ac130a2dd8 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:39.262757 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 814/1632] 10.222.0.42 () {54 vars in 912 bytes} [Wed Sep 14 01:04:39 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:04:39.264396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92917221-cc42-43f9-b6f7-10ac130a2dd8 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:39.268389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9197d4-a190-4541-8cad-bc092c6e6378 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:39.269033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9197d4-a190-4541-8cad-bc092c6e6378 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:39.269490 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9197d4-a190-4541-8cad-bc092c6e6378 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:39.271921 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9197d4-a190-4541-8cad-bc092c6e6378 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:39.297117 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9197d4-a190-4541-8cad-bc092c6e6378 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:39.299325 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92917221-cc42-43f9-b6f7-10ac130a2dd8 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:39.301164 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9197d4-a190-4541-8cad-bc092c6e6378 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:39.310394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2b691-02e4-47e0-80f8-61d96acac405 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:39.311512 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2b691-02e4-47e0-80f8-61d96acac405 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:39.312408 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2b691-02e4-47e0-80f8-61d96acac405 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:39.312700 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2b691-02e4-47e0-80f8-61d96acac405 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:39.336097 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 819/1633] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:04:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:04:39.340020 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b46b3-4750-4675-a7c8-76cc4b5dd055 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:39.340898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b46b3-4750-4675-a7c8-76cc4b5dd055 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:39.341772 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b46b3-4750-4675-a7c8-76cc4b5dd055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:39.347191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2b691-02e4-47e0-80f8-61d96acac405 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:39.350379 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2b691-02e4-47e0-80f8-61d96acac405 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:39.389428 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 815/1634] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:04:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:04:39.399848 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-895b46b3-4750-4675-a7c8-76cc4b5dd055 None None] MFA Rules not processed for user `5e0fe2e84f6748c992c9beedf2f5d49b`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:39.404843 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-895b46b3-4750-4675-a7c8-76cc4b5dd055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:39.426477 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 820/1635] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 01:04:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:39.441149 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2b691-02e4-47e0-80f8-61d96acac405 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:39.452520 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2b691-02e4-47e0-80f8-61d96acac405 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:39.457969 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a3483-cc64-405a-8a8f-866608651408 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:39.460952 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a3483-cc64-405a-8a8f-866608651408 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:39.461668 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a3483-cc64-405a-8a8f-866608651408 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:39.461947 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a3483-cc64-405a-8a8f-866608651408 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:39.481120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a3483-cc64-405a-8a8f-866608651408 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:39.482937 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a3483-cc64-405a-8a8f-866608651408 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:39.498546 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 816/1636] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:04:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:04:41.003774 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b46b3-4750-4675-a7c8-76cc4b5dd055 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:41.187122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b46b3-4750-4675-a7c8-76cc4b5dd055 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:41.192773 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd21a59-b89a-419f-8715-a27be9d511b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:41.194053 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd21a59-b89a-419f-8715-a27be9d511b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:41.194445 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd21a59-b89a-419f-8715-a27be9d511b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:41.194937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd21a59-b89a-419f-8715-a27be9d511b4 admin admin] PATH_INFO: `/v3/users/e478c96e7ba640ebb95edfec9b029c4c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:41.196536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd21a59-b89a-419f-8715-a27be9d511b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e478c96e7ba640ebb95edfec9b029c4c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:41.197686 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: WARNING py.warnings [None req-3cd21a59-b89a-419f-8715-a27be9d511b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 01:04:41.197686 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: warnings.warn(msg) Sep 14 01:04:41.197686 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: Sep 14 01:04:41.199170 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd21a59-b89a-419f-8715-a27be9d511b4 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:41.385329 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3cd21a59-b89a-419f-8715-a27be9d511b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e478c96e7ba640ebb95edfec9b029c4c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:41.385329 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3cd21a59-b89a-419f-8715-a27be9d511b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e478c96e7ba640ebb95edfec9b029c4c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:41.385329 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3cd21a59-b89a-419f-8715-a27be9d511b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e478c96e7ba640ebb95edfec9b029c4c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:41.385329 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3cd21a59-b89a-419f-8715-a27be9d511b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e478c96e7ba640ebb95edfec9b029c4c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:41.416750 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3cd21a59-b89a-419f-8715-a27be9d511b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e478c96e7ba640ebb95edfec9b029c4c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:41.447608 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 821/1637] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:04:41 2022] DELETE /identity/v3/users/e478c96e7ba640ebb95edfec9b029c4c => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:41.452886 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3c020-1f03-48ce-8535-4515e9e2d802 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:41.453454 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3c020-1f03-48ce-8535-4515e9e2d802 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:41.453454 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3c020-1f03-48ce-8535-4515e9e2d802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:41.518118 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-0fe3c020-1f03-48ce-8535-4515e9e2d802 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:41.519435 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0fe3c020-1f03-48ce-8535-4515e9e2d802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:41.641403 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 817/1638] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:41.670543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd21a59-b89a-419f-8715-a27be9d511b4 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:41.672417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3cd21a59-b89a-419f-8715-a27be9d511b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:41.715344 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:41.766257 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd21a59-b89a-419f-8715-a27be9d511b4 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:41.770730 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af286d09-0dc0-426c-b2f3-c31870495977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:41.771833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af286d09-0dc0-426c-b2f3-c31870495977 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:41.772405 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af286d09-0dc0-426c-b2f3-c31870495977 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:41.773443 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af286d09-0dc0-426c-b2f3-c31870495977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:41.787791 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af286d09-0dc0-426c-b2f3-c31870495977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:41.790797 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af286d09-0dc0-426c-b2f3-c31870495977 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:41.812914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 822/1639] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:04:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:04:43.464679 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3c020-1f03-48ce-8535-4515e9e2d802 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:43.467209 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0fe3c020-1f03-48ce-8535-4515e9e2d802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:43.500971 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3c020-1f03-48ce-8535-4515e9e2d802 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:43.518218 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d0687-0a83-4533-8cf5-92b0414422cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:43.519063 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d0687-0a83-4533-8cf5-92b0414422cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:43.519343 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d0687-0a83-4533-8cf5-92b0414422cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:43.520038 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d0687-0a83-4533-8cf5-92b0414422cf admin admin] PATH_INFO: `/v3/projects/277d73e83c4047c098c5e5e38c057b7a` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:43.522436 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0d0687-0a83-4533-8cf5-92b0414422cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=277d73e83c4047c098c5e5e38c057b7a)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:43.524265 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: WARNING py.warnings [None req-4f0d0687-0a83-4533-8cf5-92b0414422cf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 01:04:43.524265 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: warnings.warn(msg) Sep 14 01:04:43.524265 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: Sep 14 01:04:43.525457 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0d0687-0a83-4533-8cf5-92b0414422cf admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:43.611595 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:43.731257 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4f0d0687-0a83-4533-8cf5-92b0414422cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '277d73e83c4047c098c5e5e38c057b7a', 'request_id': 'req-4f0d0687-0a83-4533-8cf5-92b0414422cf'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:43.751141 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4f0d0687-0a83-4533-8cf5-92b0414422cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '277d73e83c4047c098c5e5e38c057b7a', 'request_id': 'req-4f0d0687-0a83-4533-8cf5-92b0414422cf'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:43.761333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4f0d0687-0a83-4533-8cf5-92b0414422cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '277d73e83c4047c098c5e5e38c057b7a', 'request_id': 'req-4f0d0687-0a83-4533-8cf5-92b0414422cf'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:43.839525 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4f0d0687-0a83-4533-8cf5-92b0414422cf admin admin] The token cache is being invalidate because project 277d73e83c4047c098c5e5e38c057b7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:04:43.839851 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4f0d0687-0a83-4533-8cf5-92b0414422cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:43.840523 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4f0d0687-0a83-4533-8cf5-92b0414422cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:43.842295 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 818/1640] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:04:43 2022] DELETE /identity/v3/projects/277d73e83c4047c098c5e5e38c057b7a => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:43.846632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af286d09-0dc0-426c-b2f3-c31870495977 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:43.857055 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af286d09-0dc0-426c-b2f3-c31870495977 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:44.021009 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cb99e-4c0a-416d-9cf3-1e22e39d9e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:44.021821 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cb99e-4c0a-416d-9cf3-1e22e39d9e11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:44.022109 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cb99e-4c0a-416d-9cf3-1e22e39d9e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:44.022457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cb99e-4c0a-416d-9cf3-1e22e39d9e11 admin admin] PATH_INFO: `/v3/users/1404f53b904946d4981f35880b535d70` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:44.024603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738cb99e-4c0a-416d-9cf3-1e22e39d9e11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1404f53b904946d4981f35880b535d70)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:44.026568 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738cb99e-4c0a-416d-9cf3-1e22e39d9e11 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:44.141214 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-738cb99e-4c0a-416d-9cf3-1e22e39d9e11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1404f53b904946d4981f35880b535d70'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:44.147294 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-738cb99e-4c0a-416d-9cf3-1e22e39d9e11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1404f53b904946d4981f35880b535d70'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:44.167161 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-738cb99e-4c0a-416d-9cf3-1e22e39d9e11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1404f53b904946d4981f35880b535d70'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:44.167823 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-738cb99e-4c0a-416d-9cf3-1e22e39d9e11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1404f53b904946d4981f35880b535d70'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:44.179916 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-738cb99e-4c0a-416d-9cf3-1e22e39d9e11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1404f53b904946d4981f35880b535d70'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:44.230495 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 823/1641] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:04:43 2022] DELETE /identity/v3/users/1404f53b904946d4981f35880b535d70 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:04:44.615868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d0687-0a83-4533-8cf5-92b0414422cf admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:44.617818 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4f0d0687-0a83-4533-8cf5-92b0414422cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:44.644214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d0687-0a83-4533-8cf5-92b0414422cf admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:44.799216 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2412b-c191-4240-a59c-4eaf67523617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:44.800930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2412b-c191-4240-a59c-4eaf67523617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:44.801814 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2412b-c191-4240-a59c-4eaf67523617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:44.804001 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2412b-c191-4240-a59c-4eaf67523617 admin admin] PATH_INFO: `/v3/users/0ded01b9a7c44e89a6ec9d0f8d0bd1a1` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:44.807145 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e2412b-c191-4240-a59c-4eaf67523617 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ded01b9a7c44e89a6ec9d0f8d0bd1a1)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:44.810438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e2412b-c191-4240-a59c-4eaf67523617 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:44.923430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-15e2412b-c191-4240-a59c-4eaf67523617 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ded01b9a7c44e89a6ec9d0f8d0bd1a1'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:44.924326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-15e2412b-c191-4240-a59c-4eaf67523617 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ded01b9a7c44e89a6ec9d0f8d0bd1a1'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:44.938170 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-15e2412b-c191-4240-a59c-4eaf67523617 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ded01b9a7c44e89a6ec9d0f8d0bd1a1'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:44.951451 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-15e2412b-c191-4240-a59c-4eaf67523617 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ded01b9a7c44e89a6ec9d0f8d0bd1a1'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:44.954017 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-15e2412b-c191-4240-a59c-4eaf67523617 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ded01b9a7c44e89a6ec9d0f8d0bd1a1'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:44.990675 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 819/1642] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:04:44 2022] DELETE /identity/v3/users/0ded01b9a7c44e89a6ec9d0f8d0bd1a1 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:44.995944 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28054f4c-e243-42e9-a930-480fd04e0291 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:44.996768 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28054f4c-e243-42e9-a930-480fd04e0291 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:44.997365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28054f4c-e243-42e9-a930-480fd04e0291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:45.019490 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-28054f4c-e243-42e9-a930-480fd04e0291 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:45.021227 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-28054f4c-e243-42e9-a930-480fd04e0291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:45.082566 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:45.150236 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 824/1643] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:04:45.192239 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2412b-c191-4240-a59c-4eaf67523617 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:45.195266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-15e2412b-c191-4240-a59c-4eaf67523617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:45.279693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:45.365609 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2412b-c191-4240-a59c-4eaf67523617 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:45.370458 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd51c5-122f-451c-a8e6-e5511e2d8f20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:45.371334 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd51c5-122f-451c-a8e6-e5511e2d8f20 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:45.371799 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd51c5-122f-451c-a8e6-e5511e2d8f20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:45.372228 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd51c5-122f-451c-a8e6-e5511e2d8f20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:45.384756 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd51c5-122f-451c-a8e6-e5511e2d8f20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:45.386503 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd51c5-122f-451c-a8e6-e5511e2d8f20 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:45.407870 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 820/1644] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:04:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:04:46.773231 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28054f4c-e243-42e9-a930-480fd04e0291 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:46.774743 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-28054f4c-e243-42e9-a930-480fd04e0291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:46.801442 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28054f4c-e243-42e9-a930-480fd04e0291 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:46.818732 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a2fb5-ec41-460d-91f9-80c21d8639d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:46.819544 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a2fb5-ec41-460d-91f9-80c21d8639d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:46.819796 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a2fb5-ec41-460d-91f9-80c21d8639d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:46.820074 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a2fb5-ec41-460d-91f9-80c21d8639d5 admin admin] PATH_INFO: `/v3/projects/3a1767d9b3cd495bb8e1e9fe4ecb0946` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:46.822122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a2fb5-ec41-460d-91f9-80c21d8639d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a1767d9b3cd495bb8e1e9fe4ecb0946)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:46.828629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a2fb5-ec41-460d-91f9-80c21d8639d5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:46.894803 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b72a2fb5-ec41-460d-91f9-80c21d8639d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a1767d9b3cd495bb8e1e9fe4ecb0946', 'request_id': 'req-b72a2fb5-ec41-460d-91f9-80c21d8639d5'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:46.914709 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b72a2fb5-ec41-460d-91f9-80c21d8639d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a1767d9b3cd495bb8e1e9fe4ecb0946', 'request_id': 'req-b72a2fb5-ec41-460d-91f9-80c21d8639d5'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:46.923093 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b72a2fb5-ec41-460d-91f9-80c21d8639d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a1767d9b3cd495bb8e1e9fe4ecb0946', 'request_id': 'req-b72a2fb5-ec41-460d-91f9-80c21d8639d5'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:46.938274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b72a2fb5-ec41-460d-91f9-80c21d8639d5 admin admin] The token cache is being invalidate because project 3a1767d9b3cd495bb8e1e9fe4ecb0946 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:04:46.938663 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b72a2fb5-ec41-460d-91f9-80c21d8639d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:46.939968 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b72a2fb5-ec41-460d-91f9-80c21d8639d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:46.945237 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 825/1645] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:04:46 2022] DELETE /identity/v3/projects/3a1767d9b3cd495bb8e1e9fe4ecb0946 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:46.947133 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfec37-5f40-4d10-889d-e8548f50de7b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:46.947694 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfec37-5f40-4d10-889d-e8548f50de7b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:46.947694 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfec37-5f40-4d10-889d-e8548f50de7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:46.987890 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-cfcfec37-5f40-4d10-889d-e8548f50de7b None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:46.989142 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-cfcfec37-5f40-4d10-889d-e8548f50de7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:47.135923 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 821/1646] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:47.137127 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5501935-e6c0-47c7-a3f2-cce080dcf517 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:47.137719 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5501935-e6c0-47c7-a3f2-cce080dcf517 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:47.138116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5501935-e6c0-47c7-a3f2-cce080dcf517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:47.195329 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f5501935-e6c0-47c7-a3f2-cce080dcf517 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:47.196891 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f5501935-e6c0-47c7-a3f2-cce080dcf517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:47.218598 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfec37-5f40-4d10-889d-e8548f50de7b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:47.219196 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 826/1647] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:47.232120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfec37-5f40-4d10-889d-e8548f50de7b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:47.241156 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cedf09-bced-4813-83cf-0b95c7bcabc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:47.242506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cedf09-bced-4813-83cf-0b95c7bcabc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:47.242994 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cedf09-bced-4813-83cf-0b95c7bcabc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:47.243367 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cedf09-bced-4813-83cf-0b95c7bcabc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:47.244060 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cedf09-bced-4813-83cf-0b95c7bcabc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:04:47.249417 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cedf09-bced-4813-83cf-0b95c7bcabc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:47.257297 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cedf09-bced-4813-83cf-0b95c7bcabc0 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:47.279191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 822/1648] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:04:47.283737 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5501935-e6c0-47c7-a3f2-cce080dcf517 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:47.300667 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5501935-e6c0-47c7-a3f2-cce080dcf517 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:47.313039 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae9217-92e8-40b3-af15-8cb6827a8a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:47.314447 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae9217-92e8-40b3-af15-8cb6827a8a1d admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:47.315997 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae9217-92e8-40b3-af15-8cb6827a8a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:47.316401 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae9217-92e8-40b3-af15-8cb6827a8a1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:47.317181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae9217-92e8-40b3-af15-8cb6827a8a1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:47.325345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae9217-92e8-40b3-af15-8cb6827a8a1d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:47.411480 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 827/1649] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:04:47 2022] POST /identity/v3/projects => generated 409 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:47.416603 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cedf09-bced-4813-83cf-0b95c7bcabc0 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:47.438255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cedf09-bced-4813-83cf-0b95c7bcabc0 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:47.499399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae9217-92e8-40b3-af15-8cb6827a8a1d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:47.502425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7eae9217-92e8-40b3-af15-8cb6827a8a1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:47.589340 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4882d7d-a947-4d7b-ae76-a7524d1bc370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:47.589890 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4882d7d-a947-4d7b-ae76-a7524d1bc370 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:47.590171 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4882d7d-a947-4d7b-ae76-a7524d1bc370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:47.590446 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4882d7d-a947-4d7b-ae76-a7524d1bc370 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:47.591090 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4882d7d-a947-4d7b-ae76-a7524d1bc370 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:47.593079 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4882d7d-a947-4d7b-ae76-a7524d1bc370 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:47.642362 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae9217-92e8-40b3-af15-8cb6827a8a1d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:47.651308 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e94112e-377d-4591-a00c-47f79a710f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:47.652356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94112e-377d-4591-a00c-47f79a710f99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:47.652783 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94112e-377d-4591-a00c-47f79a710f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:47.653343 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94112e-377d-4591-a00c-47f79a710f99 admin admin] PATH_INFO: `/v3/projects/437899e27b144fcea3b6c06acc7df525` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:47.656949 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e94112e-377d-4591-a00c-47f79a710f99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=437899e27b144fcea3b6c06acc7df525)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:47.657106 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 823/1650] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:04:47 2022] POST /identity/v3/users => generated 305 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:47.657748 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e94112e-377d-4591-a00c-47f79a710f99 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:47.658084 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4882d7d-a947-4d7b-ae76-a7524d1bc370 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:47.688235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4882d7d-a947-4d7b-ae76-a7524d1bc370 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:47.703957 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b800d8-8867-4d5a-b190-7d34ee218a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:47.704599 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b800d8-8867-4d5a-b190-7d34ee218a17 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:47.704887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b800d8-8867-4d5a-b190-7d34ee218a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:47.705165 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b800d8-8867-4d5a-b190-7d34ee218a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:47.705822 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b800d8-8867-4d5a-b190-7d34ee218a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:47.706140 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:47.707517 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b800d8-8867-4d5a-b190-7d34ee218a17 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:47.723617 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1e94112e-377d-4591-a00c-47f79a710f99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '437899e27b144fcea3b6c06acc7df525', 'request_id': 'req-1e94112e-377d-4591-a00c-47f79a710f99'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:47.734202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1e94112e-377d-4591-a00c-47f79a710f99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '437899e27b144fcea3b6c06acc7df525', 'request_id': 'req-1e94112e-377d-4591-a00c-47f79a710f99'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:47.739295 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 824/1651] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:04:47.743810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1e94112e-377d-4591-a00c-47f79a710f99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '437899e27b144fcea3b6c06acc7df525', 'request_id': 'req-1e94112e-377d-4591-a00c-47f79a710f99'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:47.744111 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b800d8-8867-4d5a-b190-7d34ee218a17 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:47.764585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1e94112e-377d-4591-a00c-47f79a710f99 admin admin] The token cache is being invalidate because project 437899e27b144fcea3b6c06acc7df525 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:04:47.765194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1e94112e-377d-4591-a00c-47f79a710f99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:47.766018 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1e94112e-377d-4591-a00c-47f79a710f99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:47.766688 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b800d8-8867-4d5a-b190-7d34ee218a17 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:47.768601 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 828/1652] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:04:47 2022] DELETE /identity/v3/projects/437899e27b144fcea3b6c06acc7df525 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:47.774240 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba36e10-2459-4030-b850-6116db3c505e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:47.774875 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba36e10-2459-4030-b850-6116db3c505e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:47.774875 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba36e10-2459-4030-b850-6116db3c505e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:47.801634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-dba36e10-2459-4030-b850-6116db3c505e None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:47.804802 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dba36e10-2459-4030-b850-6116db3c505e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:47.865017 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:47.883402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 829/1653] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:47.891393 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea981a-cffc-402d-8e78-8bb346dee1a2 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:47.892616 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea981a-cffc-402d-8e78-8bb346dee1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:47.893169 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea981a-cffc-402d-8e78-8bb346dee1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:47.930497 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e707dca-7a4d-4e6a-8900-e1fd97d59e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:47.931142 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e707dca-7a4d-4e6a-8900-e1fd97d59e17 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:47.931429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e707dca-7a4d-4e6a-8900-e1fd97d59e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:47.931735 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e707dca-7a4d-4e6a-8900-e1fd97d59e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:47.932441 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e707dca-7a4d-4e6a-8900-e1fd97d59e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:47.934100 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e707dca-7a4d-4e6a-8900-e1fd97d59e17 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:47.946256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-71ea981a-cffc-402d-8e78-8bb346dee1a2 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:47.947397 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-71ea981a-cffc-402d-8e78-8bb346dee1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:47.961837 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 825/1654] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:47.964122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 830/1655] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:04:47.967874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e707dca-7a4d-4e6a-8900-e1fd97d59e17 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:47.968566 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea981a-cffc-402d-8e78-8bb346dee1a2 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:47.978212 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4e707dca-7a4d-4e6a-8900-e1fd97d59e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:47.981111 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea981a-cffc-402d-8e78-8bb346dee1a2 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:47.989296 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9085d-3a59-41a9-a848-15ee28a34199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:47.990280 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9085d-3a59-41a9-a848-15ee28a34199 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:47.990766 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9085d-3a59-41a9-a848-15ee28a34199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:47.992444 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9085d-3a59-41a9-a848-15ee28a34199 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:47.993668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9085d-3a59-41a9-a848-15ee28a34199 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:04:47.994248 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9085d-3a59-41a9-a848-15ee28a34199 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:47.996709 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9085d-3a59-41a9-a848-15ee28a34199 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:48.006336 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e707dca-7a4d-4e6a-8900-e1fd97d59e17 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:48.020045 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 831/1656] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:04:48.021390 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9085d-3a59-41a9-a848-15ee28a34199 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:48.024846 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3af30-24e3-46e6-af52-1e5ca34a50bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:48.025736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3af30-24e3-46e6-af52-1e5ca34a50bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:48.028757 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3af30-24e3-46e6-af52-1e5ca34a50bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:48.029209 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3af30-24e3-46e6-af52-1e5ca34a50bd admin admin] PATH_INFO: `/v3/projects/d9c3fbccc03a4770a9e6a6412cca4212/users/7baadb3685204433ba57317a022b2e29/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:48.036638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9085d-3a59-41a9-a848-15ee28a34199 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:48.043444 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c0a69-5634-4d21-bbc7-666f12283af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:48.044413 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c0a69-5634-4d21-bbc7-666f12283af0 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:48.044859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c0a69-5634-4d21-bbc7-666f12283af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:48.045255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c0a69-5634-4d21-bbc7-666f12283af0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:48.046883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c0a69-5634-4d21-bbc7-666f12283af0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:48.049378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c0a69-5634-4d21-bbc7-666f12283af0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:48.065988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c3af30-24e3-46e6-af52-1e5ca34a50bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9c3fbccc03a4770a9e6a6412cca4212, user_id=7baadb3685204433ba57317a022b2e29, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:48.069031 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c3af30-24e3-46e6-af52-1e5ca34a50bd admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:48.093198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 826/1657] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:04:47 2022] PUT /identity/v3/projects/d9c3fbccc03a4770a9e6a6412cca4212/users/7baadb3685204433ba57317a022b2e29/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:48.106729 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 832/1658] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:04:48 2022] POST /identity/v3/projects => generated 363 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:48.108307 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c0a69-5634-4d21-bbc7-666f12283af0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:48.108666 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1e626-b323-461f-9c8f-cbb55895cb5b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:48.109127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1e626-b323-461f-9c8f-cbb55895cb5b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:48.109560 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1e626-b323-461f-9c8f-cbb55895cb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:48.153614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-85e1e626-b323-461f-9c8f-cbb55895cb5b None None] MFA Rules not processed for user `7baadb3685204433ba57317a022b2e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:48.155496 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-85e1e626-b323-461f-9c8f-cbb55895cb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:48.243608 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c0a69-5634-4d21-bbc7-666f12283af0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:48.258773 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4103a6ea-c20b-41a4-adcf-6da379d75053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:48.260290 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103a6ea-c20b-41a4-adcf-6da379d75053 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:48.260583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103a6ea-c20b-41a4-adcf-6da379d75053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:48.260871 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4103a6ea-c20b-41a4-adcf-6da379d75053 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:48.261524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4103a6ea-c20b-41a4-adcf-6da379d75053 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:48.263760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 827/1659] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:48 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:48.264146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4103a6ea-c20b-41a4-adcf-6da379d75053 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:48.268747 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1e626-b323-461f-9c8f-cbb55895cb5b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:48.295443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1e626-b323-461f-9c8f-cbb55895cb5b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:48.310162 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf27cd-bf1a-4253-9c20-b0cb7d357acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:48.312413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf27cd-bf1a-4253-9c20-b0cb7d357acd admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:48.312839 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf27cd-bf1a-4253-9c20-b0cb7d357acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:48.313201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf27cd-bf1a-4253-9c20-b0cb7d357acd admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:48.315500 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf27cd-bf1a-4253-9c20-b0cb7d357acd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:48.323938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf27cd-bf1a-4253-9c20-b0cb7d357acd admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:48.328641 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 833/1660] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:04:48 2022] POST /identity/v3/users => generated 282 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:48.332598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4103a6ea-c20b-41a4-adcf-6da379d75053 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:48.377230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4103a6ea-c20b-41a4-adcf-6da379d75053 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:48.383108 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1147a-5e75-4b8a-b8b7-d6c295ec11c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:48.383861 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1147a-5e75-4b8a-b8b7-d6c295ec11c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:48.384218 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1147a-5e75-4b8a-b8b7-d6c295ec11c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:48.384571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1147a-5e75-4b8a-b8b7-d6c295ec11c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:48.385304 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1147a-5e75-4b8a-b8b7-d6c295ec11c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:48.387009 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1147a-5e75-4b8a-b8b7-d6c295ec11c5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:48.393204 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 828/1661] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:04:48 2022] POST /identity/v3/projects => generated 409 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:04:48.396883 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf27cd-bf1a-4253-9c20-b0cb7d357acd admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:48.409095 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 834/1662] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:48.410721 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1147a-5e75-4b8a-b8b7-d6c295ec11c5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:48.428351 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:48.435413 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1147a-5e75-4b8a-b8b7-d6c295ec11c5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:48.558708 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf27cd-bf1a-4253-9c20-b0cb7d357acd admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:48.560979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55687b-90d5-40cf-88f2-6d1066734d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:48.561866 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55687b-90d5-40cf-88f2-6d1066734d02 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:48.562367 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55687b-90d5-40cf-88f2-6d1066734d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:48.562862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55687b-90d5-40cf-88f2-6d1066734d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:48.563757 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb55687b-90d5-40cf-88f2-6d1066734d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:48.565632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb55687b-90d5-40cf-88f2-6d1066734d02 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:48.570232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454c9b1-5566-4a6f-a124-69d3fc8bf357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:48.570922 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454c9b1-5566-4a6f-a124-69d3fc8bf357 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:48.571305 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454c9b1-5566-4a6f-a124-69d3fc8bf357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:48.571729 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454c9b1-5566-4a6f-a124-69d3fc8bf357 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:48.572399 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0454c9b1-5566-4a6f-a124-69d3fc8bf357 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:48.586732 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0454c9b1-5566-4a6f-a124-69d3fc8bf357 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:48.598971 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 835/1663] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:48.600695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55687b-90d5-40cf-88f2-6d1066734d02 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:48.630359 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55687b-90d5-40cf-88f2-6d1066734d02 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:48.634442 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 829/1664] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:04:48 2022] POST /identity/v3/users => generated 311 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:48.637582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2e48c-b5d5-4630-96c0-6335606ec241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:48.638472 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2e48c-b5d5-4630-96c0-6335606ec241 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:48.638925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2e48c-b5d5-4630-96c0-6335606ec241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:48.639246 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454c9b1-5566-4a6f-a124-69d3fc8bf357 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:48.639525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2e48c-b5d5-4630-96c0-6335606ec241 admin admin] PATH_INFO: `/v3/projects/9406307e226f40e7953ab4dad717bae5/users/49c0500eb9c84f3595cb1558553fd378/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:48.657929 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454c9b1-5566-4a6f-a124-69d3fc8bf357 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:48.664401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382d3a7-ccf4-4c50-850e-75edaf95308a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:48.665347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382d3a7-ccf4-4c50-850e-75edaf95308a admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:48.665754 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382d3a7-ccf4-4c50-850e-75edaf95308a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:48.666102 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382d3a7-ccf4-4c50-850e-75edaf95308a admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:48.666830 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1382d3a7-ccf4-4c50-850e-75edaf95308a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:48.669306 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1382d3a7-ccf4-4c50-850e-75edaf95308a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:48.671732 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:48.681648 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2e48c-b5d5-4630-96c0-6335606ec241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9406307e226f40e7953ab4dad717bae5, user_id=49c0500eb9c84f3595cb1558553fd378, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:48.684768 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2e48c-b5d5-4630-96c0-6335606ec241 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:48.690747 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 830/1665] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:48.695334 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382d3a7-ccf4-4c50-850e-75edaf95308a admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:48.714697 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382d3a7-ccf4-4c50-850e-75edaf95308a admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:48.716793 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 836/1666] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:04:48 2022] PUT /identity/v3/projects/9406307e226f40e7953ab4dad717bae5/users/49c0500eb9c84f3595cb1558553fd378/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:48.718337 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea11da8-2a38-44f1-b308-f1349cb06693 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:48.718633 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea11da8-2a38-44f1-b308-f1349cb06693 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:48.718916 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea11da8-2a38-44f1-b308-f1349cb06693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:48.789209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-bea11da8-2a38-44f1-b308-f1349cb06693 None None] MFA Rules not processed for user `49c0500eb9c84f3595cb1558553fd378`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:48.790484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bea11da8-2a38-44f1-b308-f1349cb06693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:48.834920 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29d8b2-515d-4427-8251-3afd157b3b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:48.836377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29d8b2-515d-4427-8251-3afd157b3b8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:48.837517 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29d8b2-515d-4427-8251-3afd157b3b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:48.837517 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29d8b2-515d-4427-8251-3afd157b3b8a admin admin] PATH_INFO: `/v3/projects/3b088a56350e46f6bbf3bf8023b157c4/users/7fa1e2f344b54be591e5efea7447d7e7/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:48.862592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a29d8b2-515d-4427-8251-3afd157b3b8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b088a56350e46f6bbf3bf8023b157c4, user_id=7fa1e2f344b54be591e5efea7447d7e7, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:48.870252 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a29d8b2-515d-4427-8251-3afd157b3b8a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:48.908458 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 831/1667] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:04:48 2022] PUT /identity/v3/projects/3b088a56350e46f6bbf3bf8023b157c4/users/7fa1e2f344b54be591e5efea7447d7e7/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:48.912987 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29d8b2-515d-4427-8251-3afd157b3b8a admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:48.930933 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 837/1668] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:48.934721 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea11da8-2a38-44f1-b308-f1349cb06693 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:48.969097 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea11da8-2a38-44f1-b308-f1349cb06693 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:49.110136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f9a7e-351d-4c78-94d7-87e399685f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:49.111254 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f9a7e-351d-4c78-94d7-87e399685f5d admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:49.111254 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f9a7e-351d-4c78-94d7-87e399685f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:49.111439 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f9a7e-351d-4c78-94d7-87e399685f5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:49.112214 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f9a7e-351d-4c78-94d7-87e399685f5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:49.114185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f9a7e-351d-4c78-94d7-87e399685f5d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:49.126078 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29d8b2-515d-4427-8251-3afd157b3b8a admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:49.137659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:49.143087 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939532d6-8e24-497a-a7ee-c2bef2021995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:49.146322 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939532d6-8e24-497a-a7ee-c2bef2021995 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:49.166764 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939532d6-8e24-497a-a7ee-c2bef2021995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:49.167430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939532d6-8e24-497a-a7ee-c2bef2021995 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:49.175979 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939532d6-8e24-497a-a7ee-c2bef2021995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:49.178208 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939532d6-8e24-497a-a7ee-c2bef2021995 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:49.199615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 838/1669] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:04:48 2022] POST /identity/v3/projects => generated 363 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:49.201736 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f9a7e-351d-4c78-94d7-87e399685f5d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:49.219157 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 832/1670] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:49.223187 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939532d6-8e24-497a-a7ee-c2bef2021995 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:49.274159 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939532d6-8e24-497a-a7ee-c2bef2021995 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:49.429846 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7816c7-e1c9-40af-bdfc-56bab550523e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:49.435747 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7816c7-e1c9-40af-bdfc-56bab550523e admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:49.437442 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7816c7-e1c9-40af-bdfc-56bab550523e admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:49.437755 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7816c7-e1c9-40af-bdfc-56bab550523e admin admin] PATH_INFO: `/v3/projects/3b088a56350e46f6bbf3bf8023b157c4/users/7fa1e2f344b54be591e5efea7447d7e7/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:49.449580 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f9a7e-351d-4c78-94d7-87e399685f5d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:49.456147 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7816c7-e1c9-40af-bdfc-56bab550523e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b088a56350e46f6bbf3bf8023b157c4, user_id=7fa1e2f344b54be591e5efea7447d7e7, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:49.463213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e195daf-4583-49bf-a940-390162d780f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:49.465308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7816c7-e1c9-40af-bdfc-56bab550523e admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:49.466445 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e195daf-4583-49bf-a940-390162d780f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:49.474330 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e195daf-4583-49bf-a940-390162d780f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:49.474538 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e195daf-4583-49bf-a940-390162d780f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:49.475222 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e195daf-4583-49bf-a940-390162d780f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:49.497501 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e195daf-4583-49bf-a940-390162d780f2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:49.511710 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 833/1671] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:04:49 2022] PUT /identity/v3/projects/3b088a56350e46f6bbf3bf8023b157c4/users/7fa1e2f344b54be591e5efea7447d7e7/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:49.522144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a51bcf-6c91-4772-a8a5-c9ba95f084e1 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:49.523214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a51bcf-6c91-4772-a8a5-c9ba95f084e1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:49.523735 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a51bcf-6c91-4772-a8a5-c9ba95f084e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:49.557327 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 839/1672] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:04:49 2022] POST /identity/v3/users => generated 288 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:49.560949 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e195daf-4583-49bf-a940-390162d780f2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:49.567733 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:49.576619 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e195daf-4583-49bf-a940-390162d780f2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:49.585103 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-09a51bcf-6c91-4772-a8a5-c9ba95f084e1 None None] MFA Rules not processed for user `7fa1e2f344b54be591e5efea7447d7e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:49.586719 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-09a51bcf-6c91-4772-a8a5-c9ba95f084e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:49.719095 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b0868-7a6e-499c-8135-baf5529480b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:49.719917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b0868-7a6e-499c-8135-baf5529480b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:49.720295 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b0868-7a6e-499c-8135-baf5529480b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:49.720675 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b0868-7a6e-499c-8135-baf5529480b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:49.721613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b0868-7a6e-499c-8135-baf5529480b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:49.723162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b0868-7a6e-499c-8135-baf5529480b1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:49.750888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 840/1673] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:04:49.755123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b0868-7a6e-499c-8135-baf5529480b1 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:49.783881 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b0868-7a6e-499c-8135-baf5529480b1 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:49.794727 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245572f-239f-400a-b733-b2d126e3359e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:49.795544 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245572f-239f-400a-b733-b2d126e3359e admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:49.796133 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245572f-239f-400a-b733-b2d126e3359e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:49.796495 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245572f-239f-400a-b733-b2d126e3359e admin admin] PATH_INFO: `/v3/projects/657342491db0483a98707ab349b329ad/users/19e6ff6be5034efeb7f56994371f4813/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:49.833318 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245572f-239f-400a-b733-b2d126e3359e admin admin] RBAC: Authorizing `identity:create_grant(project_id=657342491db0483a98707ab349b329ad, user_id=19e6ff6be5034efeb7f56994371f4813, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:49.836906 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 834/1674] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:49 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:04:49.838259 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a51bcf-6c91-4772-a8a5-c9ba95f084e1 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:49.840014 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-09a51bcf-6c91-4772-a8a5-c9ba95f084e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:49.840554 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245572f-239f-400a-b733-b2d126e3359e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:49.869445 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 841/1675] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:04:49 2022] PUT /identity/v3/projects/657342491db0483a98707ab349b329ad/users/19e6ff6be5034efeb7f56994371f4813/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:49.873793 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245572f-239f-400a-b733-b2d126e3359e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:50.002152 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:50.004021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a51bcf-6c91-4772-a8a5-c9ba95f084e1 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:50.027153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245572f-239f-400a-b733-b2d126e3359e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:50.039504 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007b22d-bef9-402f-8394-01263e229f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:50.040432 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007b22d-bef9-402f-8394-01263e229f1a admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:50.040724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007b22d-bef9-402f-8394-01263e229f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:50.041055 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007b22d-bef9-402f-8394-01263e229f1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:50.041984 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007b22d-bef9-402f-8394-01263e229f1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:50.046236 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007b22d-bef9-402f-8394-01263e229f1a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:50.071267 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 842/1676] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:04:50.075245 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007b22d-bef9-402f-8394-01263e229f1a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:50.107810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007b22d-bef9-402f-8394-01263e229f1a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:50.125124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4219521-5133-46dd-8f38-61bac43213d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:50.125592 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4219521-5133-46dd-8f38-61bac43213d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:50.125873 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4219521-5133-46dd-8f38-61bac43213d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:50.126195 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4219521-5133-46dd-8f38-61bac43213d3 admin admin] PATH_INFO: `/v3/projects/657342491db0483a98707ab349b329ad/users/19e6ff6be5034efeb7f56994371f4813/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:50.132294 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4219521-5133-46dd-8f38-61bac43213d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=657342491db0483a98707ab349b329ad, user_id=19e6ff6be5034efeb7f56994371f4813, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:50.136178 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4219521-5133-46dd-8f38-61bac43213d3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:50.151741 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a0436-caff-4781-a90e-10240c0f3838 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:50.152424 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a0436-caff-4781-a90e-10240c0f3838 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:50.152714 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a0436-caff-4781-a90e-10240c0f3838 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:50.152995 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a0436-caff-4781-a90e-10240c0f3838 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:50.166658 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 843/1677] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:04:50 2022] PUT /identity/v3/projects/657342491db0483a98707ab349b329ad/users/19e6ff6be5034efeb7f56994371f4813/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:50.177433 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c4eef-36c9-4f1b-8309-ecaec11aa0d3 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:50.178079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c4eef-36c9-4f1b-8309-ecaec11aa0d3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:50.178368 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c4eef-36c9-4f1b-8309-ecaec11aa0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:50.243358 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-875c4eef-36c9-4f1b-8309-ecaec11aa0d3 None None] MFA Rules not processed for user `19e6ff6be5034efeb7f56994371f4813`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:50.244803 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-875c4eef-36c9-4f1b-8309-ecaec11aa0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:50.318785 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a0436-caff-4781-a90e-10240c0f3838 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:50.320935 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a0436-caff-4781-a90e-10240c0f3838 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:50.333972 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:50.362559 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 835/1678] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:04:49 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:04:50.423664 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 844/1679] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:50 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:50.436422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a0436-caff-4781-a90e-10240c0f3838 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:50.439073 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-150a0436-caff-4781-a90e-10240c0f3838 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:50.470153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a0436-caff-4781-a90e-10240c0f3838 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:50.677921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ed9a1-9a98-480a-9cd2-b4a294fabdcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:50.680351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ed9a1-9a98-480a-9cd2-b4a294fabdcd service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:50.680731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ed9a1-9a98-480a-9cd2-b4a294fabdcd service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:50.681024 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ed9a1-9a98-480a-9cd2-b4a294fabdcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:50.713410 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ed9a1-9a98-480a-9cd2-b4a294fabdcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:50.714914 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ed9a1-9a98-480a-9cd2-b4a294fabdcd service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:50.731184 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 836/1680] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:04:50 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:04:50.871339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c4eef-36c9-4f1b-8309-ecaec11aa0d3 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:50.893984 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c4eef-36c9-4f1b-8309-ecaec11aa0d3 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:50.899716 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f5a10-69c2-49fd-8bd5-daf86fafbd39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:50.900364 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f5a10-69c2-49fd-8bd5-daf86fafbd39 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:50.900647 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f5a10-69c2-49fd-8bd5-daf86fafbd39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:50.900923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f5a10-69c2-49fd-8bd5-daf86fafbd39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:51.042051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ed9a1-9a98-480a-9cd2-b4a294fabdcd service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:51.052167 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ed9a1-9a98-480a-9cd2-b4a294fabdcd service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:51.063423 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768beb7-54c4-43fd-88f4-7454b38f3979 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:51.065546 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768beb7-54c4-43fd-88f4-7454b38f3979 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:51.066135 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768beb7-54c4-43fd-88f4-7454b38f3979 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:51.066741 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768beb7-54c4-43fd-88f4-7454b38f3979 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:51.067684 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f5a10-69c2-49fd-8bd5-daf86fafbd39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:51.070748 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f5a10-69c2-49fd-8bd5-daf86fafbd39 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:51.087481 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 845/1681] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:04:50 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:04:51.201060 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4768beb7-54c4-43fd-88f4-7454b38f3979 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:51.203269 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4768beb7-54c4-43fd-88f4-7454b38f3979 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:51.244384 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 837/1682] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:04:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:04:51.984612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4d45a-04b8-41f8-8ca0-238b58637d99 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:51.985092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4d45a-04b8-41f8-8ca0-238b58637d99 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:51.985356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4d45a-04b8-41f8-8ca0-238b58637d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:52.051214 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-68f4d45a-04b8-41f8-8ca0-238b58637d99 None None] MFA Rules not processed for user `2eca12e1e0a64eeeb183a328f338f54d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:52.052749 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-68f4d45a-04b8-41f8-8ca0-238b58637d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:52.150255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:52.206834 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 846/1683] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:52.216315 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768beb7-54c4-43fd-88f4-7454b38f3979 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:52.220959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4768beb7-54c4-43fd-88f4-7454b38f3979 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:52.393209 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768beb7-54c4-43fd-88f4-7454b38f3979 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:52.405673 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa99f54-eed6-4d46-973e-252f1147d32d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:52.406641 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa99f54-eed6-4d46-973e-252f1147d32d service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:52.407027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa99f54-eed6-4d46-973e-252f1147d32d service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:52.407505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa99f54-eed6-4d46-973e-252f1147d32d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:52.422528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa99f54-eed6-4d46-973e-252f1147d32d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:52.426198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa99f54-eed6-4d46-973e-252f1147d32d service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:52.440741 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 838/1684] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:04:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:04:53.742842 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f4d45a-04b8-41f8-8ca0-238b58637d99 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:53.744467 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-68f4d45a-04b8-41f8-8ca0-238b58637d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:53.923817 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f4d45a-04b8-41f8-8ca0-238b58637d99 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:53.944435 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f3197-b542-43b8-94a2-8fa83edcf192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:53.947923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f3197-b542-43b8-94a2-8fa83edcf192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:53.949150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f3197-b542-43b8-94a2-8fa83edcf192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:53.949444 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f3197-b542-43b8-94a2-8fa83edcf192 admin admin] PATH_INFO: `/v3/users/50f5d7492f704f15b82127fa1c724f38` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:53.955542 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f3197-b542-43b8-94a2-8fa83edcf192 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50f5d7492f704f15b82127fa1c724f38)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:53.964239 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f3197-b542-43b8-94a2-8fa83edcf192 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:54.116005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3a9f3197-b542-43b8-94a2-8fa83edcf192 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50f5d7492f704f15b82127fa1c724f38'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:54.118721 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3a9f3197-b542-43b8-94a2-8fa83edcf192 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50f5d7492f704f15b82127fa1c724f38'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:54.135108 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3a9f3197-b542-43b8-94a2-8fa83edcf192 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50f5d7492f704f15b82127fa1c724f38'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:54.135840 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3a9f3197-b542-43b8-94a2-8fa83edcf192 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50f5d7492f704f15b82127fa1c724f38'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:54.154838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3a9f3197-b542-43b8-94a2-8fa83edcf192 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50f5d7492f704f15b82127fa1c724f38'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:54.210875 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 847/1685] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:04:53 2022] DELETE /identity/v3/users/50f5d7492f704f15b82127fa1c724f38 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:54.215734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ef849-27fe-4110-83b6-a2336c33bc57 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:54.216231 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ef849-27fe-4110-83b6-a2336c33bc57 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:54.216388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ef849-27fe-4110-83b6-a2336c33bc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:54.285067 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-6a9ef849-27fe-4110-83b6-a2336c33bc57 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:54.291513 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6a9ef849-27fe-4110-83b6-a2336c33bc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:54.404458 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:54.501709 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 839/1686] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:54.541213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f3197-b542-43b8-94a2-8fa83edcf192 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:54.542693 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3a9f3197-b542-43b8-94a2-8fa83edcf192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:54.731325 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f3197-b542-43b8-94a2-8fa83edcf192 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:54.747164 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f796d-1149-4626-aaba-49fb8eba11dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:54.748443 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f796d-1149-4626-aaba-49fb8eba11dd service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:54.754385 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f796d-1149-4626-aaba-49fb8eba11dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:54.756454 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f796d-1149-4626-aaba-49fb8eba11dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:54.775464 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f796d-1149-4626-aaba-49fb8eba11dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:54.777118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f796d-1149-4626-aaba-49fb8eba11dd service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:54.790738 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 848/1687] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:04:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:04:55.251840 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54571e7-a059-4b8e-98f7-700b76db8ee5 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:55.252372 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54571e7-a059-4b8e-98f7-700b76db8ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:55.252597 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54571e7-a059-4b8e-98f7-700b76db8ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:55.277625 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7aad7-ff08-461c-8c37-4cc4ac0a2697 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:55.278206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7aad7-ff08-461c-8c37-4cc4ac0a2697 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:55.278777 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7aad7-ff08-461c-8c37-4cc4ac0a2697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:55.333812 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:55.352242 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d54571e7-a059-4b8e-98f7-700b76db8ee5 None None] MFA Rules not processed for user `51768890f19140b6819ef89f5bf610c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:55.354973 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d54571e7-a059-4b8e-98f7-700b76db8ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:55.357371 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-90d7aad7-ff08-461c-8c37-4cc4ac0a2697 None None] MFA Rules not processed for user `017a9de44877436fa9531beb756c54c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:55.358655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-90d7aad7-ff08-461c-8c37-4cc4ac0a2697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:55.557803 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 849/1688] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:55.559353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52163af4-d4a1-4970-9b7f-4a79e93bca9a None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:55.559666 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52163af4-d4a1-4970-9b7f-4a79e93bca9a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:55.559949 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52163af4-d4a1-4970-9b7f-4a79e93bca9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:55.580810 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 840/1689] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:55.595574 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54571e7-a059-4b8e-98f7-700b76db8ee5 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:55.597391 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d54571e7-a059-4b8e-98f7-700b76db8ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:55.640059 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-52163af4-d4a1-4970-9b7f-4a79e93bca9a None None] MFA Rules not processed for user `017a9de44877436fa9531beb756c54c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:55.641333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-52163af4-d4a1-4970-9b7f-4a79e93bca9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:55.661969 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 850/1690] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:04:55.669489 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52163af4-d4a1-4970-9b7f-4a79e93bca9a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:55.670948 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-52163af4-d4a1-4970-9b7f-4a79e93bca9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:55.899776 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54571e7-a059-4b8e-98f7-700b76db8ee5 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:55.932733 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43816149-820f-44d3-81ae-4b78bbcc9e04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:55.933775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43816149-820f-44d3-81ae-4b78bbcc9e04 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:55.934087 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43816149-820f-44d3-81ae-4b78bbcc9e04 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:55.934373 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43816149-820f-44d3-81ae-4b78bbcc9e04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:55.948754 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52163af4-d4a1-4970-9b7f-4a79e93bca9a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:55.967460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0d96d-17c8-4f5e-b68f-348f2f7d247d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:55.968150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0d96d-17c8-4f5e-b68f-348f2f7d247d service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:55.968483 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0d96d-17c8-4f5e-b68f-348f2f7d247d service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:55.968697 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0d96d-17c8-4f5e-b68f-348f2f7d247d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:55.978231 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43816149-820f-44d3-81ae-4b78bbcc9e04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:55.981027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43816149-820f-44d3-81ae-4b78bbcc9e04 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:55.987868 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d0d96d-17c8-4f5e-b68f-348f2f7d247d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:55.991916 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d0d96d-17c8-4f5e-b68f-348f2f7d247d service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:56.015298 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 851/1691] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:04:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:04:56.070682 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 841/1692] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:04:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:04:56.260652 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbee414-27fa-4c3a-90bf-51467c95a938 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:56.261543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbee414-27fa-4c3a-90bf-51467c95a938 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:56.262000 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbee414-27fa-4c3a-90bf-51467c95a938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:56.326768 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-abbee414-27fa-4c3a-90bf-51467c95a938 None None] MFA Rules not processed for user `bf73eca572454b88843785d791f0b471`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:56.329838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-abbee414-27fa-4c3a-90bf-51467c95a938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:56.522716 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 852/1693] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:56.525171 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff52a13-29a8-4366-8bdb-7b4240b7984e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:56.526164 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff52a13-29a8-4366-8bdb-7b4240b7984e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:56.526669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff52a13-29a8-4366-8bdb-7b4240b7984e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:56.562381 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-eff52a13-29a8-4366-8bdb-7b4240b7984e None None] MFA Rules not processed for user `bf73eca572454b88843785d791f0b471`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:56.566020 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-eff52a13-29a8-4366-8bdb-7b4240b7984e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:56.585615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 842/1694] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:56.597941 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbee414-27fa-4c3a-90bf-51467c95a938 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:56.634952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbee414-27fa-4c3a-90bf-51467c95a938 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:56.693980 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87cd82-b5e2-46ef-b148-e4816c2eea27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:56.698060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87cd82-b5e2-46ef-b148-e4816c2eea27 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:56.699030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87cd82-b5e2-46ef-b148-e4816c2eea27 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:56.699332 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87cd82-b5e2-46ef-b148-e4816c2eea27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:56.717827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:56.734354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87cd82-b5e2-46ef-b148-e4816c2eea27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:56.743181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87cd82-b5e2-46ef-b148-e4816c2eea27 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:56.769370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff52a13-29a8-4366-8bdb-7b4240b7984e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:56.770862 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-eff52a13-29a8-4366-8bdb-7b4240b7984e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:56.774578 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 853/1695] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:04:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:04:56.818149 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff52a13-29a8-4366-8bdb-7b4240b7984e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:56.838803 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3152270f-2811-4289-a444-b05a1fd93bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:56.839480 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152270f-2811-4289-a444-b05a1fd93bcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:56.839791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152270f-2811-4289-a444-b05a1fd93bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:56.840069 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152270f-2811-4289-a444-b05a1fd93bcb admin admin] PATH_INFO: `/v3/projects/9a61a5cdab00450abcb069278b29edf3` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:56.848508 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3152270f-2811-4289-a444-b05a1fd93bcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a61a5cdab00450abcb069278b29edf3)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:56.863510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3152270f-2811-4289-a444-b05a1fd93bcb admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:57.063136 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:57.119037 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3152270f-2811-4289-a444-b05a1fd93bcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a61a5cdab00450abcb069278b29edf3', 'request_id': 'req-3152270f-2811-4289-a444-b05a1fd93bcb'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:57.141256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3152270f-2811-4289-a444-b05a1fd93bcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a61a5cdab00450abcb069278b29edf3', 'request_id': 'req-3152270f-2811-4289-a444-b05a1fd93bcb'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:57.152254 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3152270f-2811-4289-a444-b05a1fd93bcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a61a5cdab00450abcb069278b29edf3', 'request_id': 'req-3152270f-2811-4289-a444-b05a1fd93bcb'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:57.184712 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3152270f-2811-4289-a444-b05a1fd93bcb admin admin] The token cache is being invalidate because project 9a61a5cdab00450abcb069278b29edf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:04:57.185025 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3152270f-2811-4289-a444-b05a1fd93bcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:57.186167 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-3152270f-2811-4289-a444-b05a1fd93bcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:04:57.198047 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce62c97-fd9c-44e5-a1dc-c81776a44eb1 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:57.198608 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce62c97-fd9c-44e5-a1dc-c81776a44eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:57.198608 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce62c97-fd9c-44e5-a1dc-c81776a44eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:57.202121 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 843/1696] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:04:56 2022] DELETE /identity/v3/projects/9a61a5cdab00450abcb069278b29edf3 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:57.222287 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-8ce62c97-fd9c-44e5-a1dc-c81776a44eb1 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:57.223460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8ce62c97-fd9c-44e5-a1dc-c81776a44eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:57.398989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 854/1697] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:57.407233 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5061a-d3a5-45c3-85ad-be4b97660423 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:57.407528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5061a-d3a5-45c3-85ad-be4b97660423 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:57.407874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5061a-d3a5-45c3-85ad-be4b97660423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:57.477797 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-40c5061a-d3a5-45c3-85ad-be4b97660423 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:57.479985 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-40c5061a-d3a5-45c3-85ad-be4b97660423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:57.511287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 844/1698] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:57.516319 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce62c97-fd9c-44e5-a1dc-c81776a44eb1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:57.533310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce62c97-fd9c-44e5-a1dc-c81776a44eb1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:57.550980 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f988d-b58b-40ac-8d8a-60ae2ba42b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:57.551657 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f988d-b58b-40ac-8d8a-60ae2ba42b79 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:57.551952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f988d-b58b-40ac-8d8a-60ae2ba42b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:57.552216 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f988d-b58b-40ac-8d8a-60ae2ba42b79 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:57.553442 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12f988d-b58b-40ac-8d8a-60ae2ba42b79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:04:57.553790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12f988d-b58b-40ac-8d8a-60ae2ba42b79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:57.555682 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12f988d-b58b-40ac-8d8a-60ae2ba42b79 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:57.587383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 855/1699] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:04:57.590614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5061a-d3a5-45c3-85ad-be4b97660423 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:57.605873 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5061a-d3a5-45c3-85ad-be4b97660423 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:57.624368 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dfc92a-e403-449d-955d-bb170fb187ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:57.625104 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dfc92a-e403-449d-955d-bb170fb187ee admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:57.625410 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dfc92a-e403-449d-955d-bb170fb187ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:57.625686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dfc92a-e403-449d-955d-bb170fb187ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:57.626777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dfc92a-e403-449d-955d-bb170fb187ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:57.628818 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dfc92a-e403-449d-955d-bb170fb187ee admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:57.699898 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 845/1700] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:04:57 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:57.704341 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f988d-b58b-40ac-8d8a-60ae2ba42b79 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:57.724294 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f988d-b58b-40ac-8d8a-60ae2ba42b79 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:57.818187 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82e5b6-8f36-47ad-880a-33ce01ad01e9 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:57.818469 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82e5b6-8f36-47ad-880a-33ce01ad01e9 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:57.818748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82e5b6-8f36-47ad-880a-33ce01ad01e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:57.844670 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3d8fc-fcf6-4128-8462-429c0b09b95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:57.845479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3d8fc-fcf6-4128-8462-429c0b09b95e admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:57.845817 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3d8fc-fcf6-4128-8462-429c0b09b95e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:57.846090 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3d8fc-fcf6-4128-8462-429c0b09b95e admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:57.847081 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c3d8fc-fcf6-4128-8462-429c0b09b95e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:57.849166 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c3d8fc-fcf6-4128-8462-429c0b09b95e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:57.890457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 856/1701] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:04:57 2022] POST /identity/v3/users => generated 290 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:57.891592 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3d8fc-fcf6-4128-8462-429c0b09b95e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:57.903583 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-3c82e5b6-8f36-47ad-880a-33ce01ad01e9 None None] MFA Rules not processed for user `e4ad2d06f62246be8eb162257cbb4f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:57.904851 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-3c82e5b6-8f36-47ad-880a-33ce01ad01e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:57.916433 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3d8fc-fcf6-4128-8462-429c0b09b95e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:57.950930 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7295a-f553-4d33-a6e9-f29f8f2f5ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:57.952748 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7295a-f553-4d33-a6e9-f29f8f2f5ad4 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:57.954602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7295a-f553-4d33-a6e9-f29f8f2f5ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:57.955294 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7295a-f553-4d33-a6e9-f29f8f2f5ad4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:57.957751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7295a-f553-4d33-a6e9-f29f8f2f5ad4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:57.963220 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7295a-f553-4d33-a6e9-f29f8f2f5ad4 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:58.002722 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 857/1702] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:58.004506 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58e952-5dc8-4ca7-acc7-219f7e99cac6 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:58.005128 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58e952-5dc8-4ca7-acc7-219f7e99cac6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:58.005479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58e952-5dc8-4ca7-acc7-219f7e99cac6 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:58.016932 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 858/1703] 10.222.0.42 () {54 vars in 912 bytes} [Wed Sep 14 01:04:58 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:04:58.018228 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58e952-5dc8-4ca7-acc7-219f7e99cac6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:58.053655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58e952-5dc8-4ca7-acc7-219f7e99cac6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:58.073127 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab5948-9ebf-41c9-a403-3f589cefa6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:58.074628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab5948-9ebf-41c9-a403-3f589cefa6c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:58.076062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab5948-9ebf-41c9-a403-3f589cefa6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:58.076414 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab5948-9ebf-41c9-a403-3f589cefa6c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:58.078796 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab5948-9ebf-41c9-a403-3f589cefa6c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:58.085326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab5948-9ebf-41c9-a403-3f589cefa6c3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:58.099470 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 846/1704] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:57 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:58.103894 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832156a9-a70d-418b-85ce-a7676148b8b0 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:58.104970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832156a9-a70d-418b-85ce-a7676148b8b0 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:58.104970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832156a9-a70d-418b-85ce-a7676148b8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:58.121949 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 859/1705] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:58.123768 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab5948-9ebf-41c9-a403-3f589cefa6c3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:58.147989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab5948-9ebf-41c9-a403-3f589cefa6c3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:58.159880 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991f0b7-2d1f-4c32-9254-ac8e758290c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:58.161076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991f0b7-2d1f-4c32-9254-ac8e758290c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:58.161709 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991f0b7-2d1f-4c32-9254-ac8e758290c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:58.162168 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991f0b7-2d1f-4c32-9254-ac8e758290c9 admin admin] PATH_INFO: `/v3/projects/276429f3f3ba47438a625cc048752e5d/users/4c42d88828b441f1b0827f5b71f2f6da/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:58.180728 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-832156a9-a70d-418b-85ce-a7676148b8b0 None None] MFA Rules not processed for user `4d64f9bca7c546e4ba7f54c1c47d665e`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:58.182353 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-832156a9-a70d-418b-85ce-a7676148b8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:58.185832 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991f0b7-2d1f-4c32-9254-ac8e758290c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=276429f3f3ba47438a625cc048752e5d, user_id=4c42d88828b441f1b0827f5b71f2f6da, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:58.190228 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991f0b7-2d1f-4c32-9254-ac8e758290c9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:58.216195 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 860/1706] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:04:58 2022] PUT /identity/v3/projects/276429f3f3ba47438a625cc048752e5d/users/4c42d88828b441f1b0827f5b71f2f6da/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:58.234270 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28ae8f-03e8-499b-8677-b6571abea885 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:58.235460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28ae8f-03e8-499b-8677-b6571abea885 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:58.236145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28ae8f-03e8-499b-8677-b6571abea885 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:58.245112 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 861/1707] 10.222.0.42 () {54 vars in 912 bytes} [Wed Sep 14 01:04:58 2022] GET /identity => generated 270 bytes in 25 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:04:58.245112 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdb833-a10c-461c-ba1f-82cefc2109d4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:58.245305 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdb833-a10c-461c-ba1f-82cefc2109d4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:58.245901 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdb833-a10c-461c-ba1f-82cefc2109d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:58.319437 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f6cdb833-a10c-461c-ba1f-82cefc2109d4 None None] MFA Rules not processed for user `4c42d88828b441f1b0827f5b71f2f6da`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:58.320740 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f6cdb833-a10c-461c-ba1f-82cefc2109d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:58.335215 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 847/1708] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 01:04:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:58.354823 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85d83b-fcd5-4e62-877a-9ed11e865284 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:58.356771 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85d83b-fcd5-4e62-877a-9ed11e865284 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:58.356771 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85d83b-fcd5-4e62-877a-9ed11e865284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:58.373507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:58.385413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-eb85d83b-fcd5-4e62-877a-9ed11e865284 None None] MFA Rules not processed for user `4d64f9bca7c546e4ba7f54c1c47d665e`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:58.386798 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-eb85d83b-fcd5-4e62-877a-9ed11e865284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:58.402917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:58.514460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 848/1709] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 01:04:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:58.516088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85d83b-fcd5-4e62-877a-9ed11e865284 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:58.518164 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-eb85d83b-fcd5-4e62-877a-9ed11e865284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:58.543704 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 862/1710] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:58.552289 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cdb833-a10c-461c-ba1f-82cefc2109d4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:58.559843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85d83b-fcd5-4e62-877a-9ed11e865284 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:58.565111 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f50c8-9789-4fd4-941d-ea6083b0a97f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:58.565743 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f50c8-9789-4fd4-941d-ea6083b0a97f service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:58.566031 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f50c8-9789-4fd4-941d-ea6083b0a97f service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:58.566311 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f50c8-9789-4fd4-941d-ea6083b0a97f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:58.598282 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f50c8-9789-4fd4-941d-ea6083b0a97f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:58.602137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f50c8-9789-4fd4-941d-ea6083b0a97f service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:58.632871 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 849/1711] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:04:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:04:58.633986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd67bef-721d-4339-afc2-b2462a0516f6 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:58.646452 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd67bef-721d-4339-afc2-b2462a0516f6 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:58.646773 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd67bef-721d-4339-afc2-b2462a0516f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:58.705725 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-1dd67bef-721d-4339-afc2-b2462a0516f6 None None] MFA Rules not processed for user `2eca12e1e0a64eeeb183a328f338f54d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:58.706856 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1dd67bef-721d-4339-afc2-b2462a0516f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:58.722865 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cdb833-a10c-461c-ba1f-82cefc2109d4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:58.734459 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9e4e5-5049-4d85-9375-1e9c9b094bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:58.738357 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9e4e5-5049-4d85-9375-1e9c9b094bbf admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:58.738779 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9e4e5-5049-4d85-9375-1e9c9b094bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:58.739165 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9e4e5-5049-4d85-9375-1e9c9b094bbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:58.740026 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce9e4e5-5049-4d85-9375-1e9c9b094bbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:58.742280 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce9e4e5-5049-4d85-9375-1e9c9b094bbf admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:58.805661 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 863/1712] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:04:58 2022] POST /identity/v3/projects => generated 379 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:58.806813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9e4e5-5049-4d85-9375-1e9c9b094bbf admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:58.808763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3ce9e4e5-5049-4d85-9375-1e9c9b094bbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:58.932485 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 850/1713] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:04:58.933452 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd67bef-721d-4339-afc2-b2462a0516f6 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:59.007128 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd67bef-721d-4339-afc2-b2462a0516f6 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:59.046650 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9e4e5-5049-4d85-9375-1e9c9b094bbf admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:59.060787 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380bd2b9-9e09-463d-b201-130c9e85a9ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:59.061625 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bd2b9-9e09-463d-b201-130c9e85a9ed service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:59.061975 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bd2b9-9e09-463d-b201-130c9e85a9ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:59.062326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bd2b9-9e09-463d-b201-130c9e85a9ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:59.073673 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380bd2b9-9e09-463d-b201-130c9e85a9ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:59.075241 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380bd2b9-9e09-463d-b201-130c9e85a9ed service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:59.092444 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 864/1714] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:04:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:04:59.161974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba70d6-2323-4424-8dc3-3404182a7cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:59.163208 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba70d6-2323-4424-8dc3-3404182a7cf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:59.164184 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba70d6-2323-4424-8dc3-3404182a7cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:59.166219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba70d6-2323-4424-8dc3-3404182a7cf6 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:59.167051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ba70d6-2323-4424-8dc3-3404182a7cf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:59.169184 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ba70d6-2323-4424-8dc3-3404182a7cf6 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:59.214899 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 851/1715] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:04:58 2022] POST /identity/v3/users => generated 296 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:04:59.219048 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380bd2b9-9e09-463d-b201-130c9e85a9ed service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:59.239111 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380bd2b9-9e09-463d-b201-130c9e85a9ed service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:59.274607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d53bf5-97a2-4c90-8249-541c6bcfba45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:59.275415 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d53bf5-97a2-4c90-8249-541c6bcfba45 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:59.275728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d53bf5-97a2-4c90-8249-541c6bcfba45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:59.276171 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d53bf5-97a2-4c90-8249-541c6bcfba45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:59.276896 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d53bf5-97a2-4c90-8249-541c6bcfba45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:59.278761 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d53bf5-97a2-4c90-8249-541c6bcfba45 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:59.301441 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 865/1716] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:59.301901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba70d6-2323-4424-8dc3-3404182a7cf6 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:59.317721 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba70d6-2323-4424-8dc3-3404182a7cf6 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:59.331867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d35e93-416e-46cc-869d-292164f4421b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:59.332578 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d35e93-416e-46cc-869d-292164f4421b admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:59.332860 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d35e93-416e-46cc-869d-292164f4421b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:59.333743 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d35e93-416e-46cc-869d-292164f4421b admin admin] PATH_INFO: `/v3/projects/687dccccc3b54cb0a45833789544937a/users/763efea976a544869a371ebfadb7f7ad/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:59.368680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d35e93-416e-46cc-869d-292164f4421b admin admin] RBAC: Authorizing `identity:create_grant(project_id=687dccccc3b54cb0a45833789544937a, user_id=763efea976a544869a371ebfadb7f7ad, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:59.373954 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d35e93-416e-46cc-869d-292164f4421b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:59.400733 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 852/1717] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:04:59 2022] PUT /identity/v3/projects/687dccccc3b54cb0a45833789544937a/users/763efea976a544869a371ebfadb7f7ad/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:59.405411 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d53bf5-97a2-4c90-8249-541c6bcfba45 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:59.420402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d53bf5-97a2-4c90-8249-541c6bcfba45 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:59.480047 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:59.538340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15c504-eb39-4da4-a24e-156f17267095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:59.540993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15c504-eb39-4da4-a24e-156f17267095 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:59.541301 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15c504-eb39-4da4-a24e-156f17267095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:59.546012 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15c504-eb39-4da4-a24e-156f17267095 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:59.547698 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c15c504-eb39-4da4-a24e-156f17267095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:59.555491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c15c504-eb39-4da4-a24e-156f17267095 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:59.582131 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 866/1718] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:04:59.586844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d35e93-416e-46cc-869d-292164f4421b admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:59.601003 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d35e93-416e-46cc-869d-292164f4421b admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:04:59.616655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69475c6b-061f-42f8-bf98-ead89d780a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:04:59.617599 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69475c6b-061f-42f8-bf98-ead89d780a81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:59.617848 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69475c6b-061f-42f8-bf98-ead89d780a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:59.618382 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69475c6b-061f-42f8-bf98-ead89d780a81 admin admin] PATH_INFO: `/v3/projects/687dccccc3b54cb0a45833789544937a/users/763efea976a544869a371ebfadb7f7ad/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:59.619507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15c504-eb39-4da4-a24e-156f17267095 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:04:59.621299 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9c15c504-eb39-4da4-a24e-156f17267095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:59.635082 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69475c6b-061f-42f8-bf98-ead89d780a81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=687dccccc3b54cb0a45833789544937a, user_id=763efea976a544869a371ebfadb7f7ad, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:04:59.640393 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69475c6b-061f-42f8-bf98-ead89d780a81 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:04:59.705243 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 853/1719] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:04:59 2022] PUT /identity/v3/projects/687dccccc3b54cb0a45833789544937a/users/763efea976a544869a371ebfadb7f7ad/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:04:59.710670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bfd9a0-1cc4-4779-a098-1bcb07f7c454 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:04:59.712723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bfd9a0-1cc4-4779-a098-1bcb07f7c454 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:04:59.713409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bfd9a0-1cc4-4779-a098-1bcb07f7c454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:04:59.752936 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:04:59.782573 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-22bfd9a0-1cc4-4779-a098-1bcb07f7c454 None None] MFA Rules not processed for user `763efea976a544869a371ebfadb7f7ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:04:59.784641 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-22bfd9a0-1cc4-4779-a098-1bcb07f7c454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:04:59.820398 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15c504-eb39-4da4-a24e-156f17267095 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:00.067271 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24af965-1e39-492e-ba53-c85b12d43ff2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:00.068171 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24af965-1e39-492e-ba53-c85b12d43ff2 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:00.068546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24af965-1e39-492e-ba53-c85b12d43ff2 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:00.069336 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24af965-1e39-492e-ba53-c85b12d43ff2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:00.074052 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 854/1720] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:04:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:00.075362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bfd9a0-1cc4-4779-a098-1bcb07f7c454 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:00.084393 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-22bfd9a0-1cc4-4779-a098-1bcb07f7c454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:00.277495 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24af965-1e39-492e-ba53-c85b12d43ff2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:00.281020 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24af965-1e39-492e-ba53-c85b12d43ff2 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:00.310768 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bfd9a0-1cc4-4779-a098-1bcb07f7c454 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:00.312006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 867/1721] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:04:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 693 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:05:00.313669 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfdcd3f-faf8-4597-8ffb-df7a0debeb54 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:00.314247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfdcd3f-faf8-4597-8ffb-df7a0debeb54 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:00.314732 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfdcd3f-faf8-4597-8ffb-df7a0debeb54 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:00.316489 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 868/1722] 10.222.0.42 () {54 vars in 961 bytes} [Wed Sep 14 01:05:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:05:00.316672 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d17d59-f118-4a7b-a1d5-d0cab47ec644 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18798838de284c84b5b838b3530a2c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:00.317455 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d17d59-f118-4a7b-a1d5-d0cab47ec644 service placement] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:00.317593 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfdcd3f-faf8-4597-8ffb-df7a0debeb54 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:00.318019 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d17d59-f118-4a7b-a1d5-d0cab47ec644 service placement] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:00.318429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d17d59-f118-4a7b-a1d5-d0cab47ec644 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:00.319637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-edfdcd3f-faf8-4597-8ffb-df7a0debeb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:00.321410 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-90d17d59-f118-4a7b-a1d5-d0cab47ec644 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:00.334256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:00.337724 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d17d59-f118-4a7b-a1d5-d0cab47ec644 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:00.340341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d17d59-f118-4a7b-a1d5-d0cab47ec644 service placement] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:00.354739 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 855/1723] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:00.357087 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44097a-df0c-4359-8823-e54fddac4194 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:00.357578 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44097a-df0c-4359-8823-e54fddac4194 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:00.358058 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44097a-df0c-4359-8823-e54fddac4194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:00.395675 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-db44097a-df0c-4359-8823-e54fddac4194 None None] MFA Rules not processed for user `18798838de284c84b5b838b3530a2c43`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:00.399861 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-db44097a-df0c-4359-8823-e54fddac4194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:00.427349 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 856/1724] 10.222.0.42 () {58 vars in 1058 bytes} [Wed Sep 14 01:05:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:00.435886 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db44097a-df0c-4359-8823-e54fddac4194 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:00.449629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfdcd3f-faf8-4597-8ffb-df7a0debeb54 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:00.460395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b8e04-4099-4ef7-826b-b149848730d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:00.460395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b8e04-4099-4ef7-826b-b149848730d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:00.460395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b8e04-4099-4ef7-826b-b149848730d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:00.460784 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b8e04-4099-4ef7-826b-b149848730d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:00.464965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db44097a-df0c-4359-8823-e54fddac4194 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:00.472340 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b1764-4419-4fde-8464-34bbecb10d40 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18798838de284c84b5b838b3530a2c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:00.473492 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b1764-4419-4fde-8464-34bbecb10d40 service placement] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:00.474235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b1764-4419-4fde-8464-34bbecb10d40 service placement] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:00.475762 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b1764-4419-4fde-8464-34bbecb10d40 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:00.498138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b1764-4419-4fde-8464-34bbecb10d40 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:00.500182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b1764-4419-4fde-8464-34bbecb10d40 service placement] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:00.505987 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4b8e04-4099-4ef7-826b-b149848730d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:00.507614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4b8e04-4099-4ef7-826b-b149848730d1 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:00.540899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 869/1725] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:00 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:00.550572 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 857/1726] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:00.668698 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b8e04-4099-4ef7-826b-b149848730d1 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:00.692075 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b8e04-4099-4ef7-826b-b149848730d1 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:00.710478 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4776d9b-c0e2-4684-9517-b86df51fc9ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:00.718239 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4776d9b-c0e2-4684-9517-b86df51fc9ad service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:00.718536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4776d9b-c0e2-4684-9517-b86df51fc9ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:00.718892 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4776d9b-c0e2-4684-9517-b86df51fc9ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:00.898845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4776d9b-c0e2-4684-9517-b86df51fc9ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:00.900308 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4776d9b-c0e2-4684-9517-b86df51fc9ad service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:00.942622 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ee855-031b-4937-8bfe-e2b6a9192562 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:00.943127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ee855-031b-4937-8bfe-e2b6a9192562 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:00.943240 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ee855-031b-4937-8bfe-e2b6a9192562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:00.950614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 870/1727] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:01.083173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-be5ee855-031b-4937-8bfe-e2b6a9192562 None None] MFA Rules not processed for user `bf73eca572454b88843785d791f0b471`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:01.091731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-be5ee855-031b-4937-8bfe-e2b6a9192562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:01.291600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 858/1728] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:02.093414 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed683f6-33ca-4697-b7ce-7e1fa70acfda None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:02.093986 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed683f6-33ca-4697-b7ce-7e1fa70acfda None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:02.093986 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed683f6-33ca-4697-b7ce-7e1fa70acfda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:02.140437 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2ed683f6-33ca-4697-b7ce-7e1fa70acfda None None] MFA Rules not processed for user `51768890f19140b6819ef89f5bf610c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:02.141581 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2ed683f6-33ca-4697-b7ce-7e1fa70acfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:02.242180 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 871/1729] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:03.762760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fff75-4d52-4d6c-898f-7a45ecbf08ca None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:03.763295 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fff75-4d52-4d6c-898f-7a45ecbf08ca None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:03.763775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fff75-4d52-4d6c-898f-7a45ecbf08ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:03.826041 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-2c7fff75-4d52-4d6c-898f-7a45ecbf08ca None None] MFA Rules not processed for user `e4ad2d06f62246be8eb162257cbb4f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:03.827490 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2c7fff75-4d52-4d6c-898f-7a45ecbf08ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:03.880894 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 859/1730] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:03 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:05.347296 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed683f6-33ca-4697-b7ce-7e1fa70acfda None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:05.349609 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2ed683f6-33ca-4697-b7ce-7e1fa70acfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:05.450690 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:05.490429 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed683f6-33ca-4697-b7ce-7e1fa70acfda None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:05.497777 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae1aa5-9fec-491e-886c-3bec8b0515ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:05.498692 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae1aa5-9fec-491e-886c-3bec8b0515ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:05.499227 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae1aa5-9fec-491e-886c-3bec8b0515ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:05.499758 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae1aa5-9fec-491e-886c-3bec8b0515ab admin admin] PATH_INFO: `/v3/users/7baadb3685204433ba57317a022b2e29` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:05.501203 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae1aa5-9fec-491e-886c-3bec8b0515ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=7baadb3685204433ba57317a022b2e29)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:05.505217 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae1aa5-9fec-491e-886c-3bec8b0515ab admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:05.580813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-47ae1aa5-9fec-491e-886c-3bec8b0515ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7baadb3685204433ba57317a022b2e29'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:05.597156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-47ae1aa5-9fec-491e-886c-3bec8b0515ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7baadb3685204433ba57317a022b2e29'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:05.619244 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-47ae1aa5-9fec-491e-886c-3bec8b0515ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7baadb3685204433ba57317a022b2e29'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:05.620173 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-47ae1aa5-9fec-491e-886c-3bec8b0515ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7baadb3685204433ba57317a022b2e29'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:05.640875 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-47ae1aa5-9fec-491e-886c-3bec8b0515ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7baadb3685204433ba57317a022b2e29'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:05.683598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 872/1731] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:05 2022] DELETE /identity/v3/users/7baadb3685204433ba57317a022b2e29 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:05.687888 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a00d91-8044-4395-a2b4-1a5887026bff None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:05.688364 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a00d91-8044-4395-a2b4-1a5887026bff None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:05.688466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a00d91-8044-4395-a2b4-1a5887026bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:05.702081 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:05.742417 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-13a00d91-8044-4395-a2b4-1a5887026bff None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:05.748524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-13a00d91-8044-4395-a2b4-1a5887026bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:05.899854 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efe829-92a4-463b-a5fc-368af222b5b4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:05.900233 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efe829-92a4-463b-a5fc-368af222b5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:05.900589 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efe829-92a4-463b-a5fc-368af222b5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:05.938573 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 860/1732] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:05.949059 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a00d91-8044-4395-a2b4-1a5887026bff None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:05.957831 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-13a00d91-8044-4395-a2b4-1a5887026bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:06.117738 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-59efe829-92a4-463b-a5fc-368af222b5b4 None None] MFA Rules not processed for user `ceccc909529042539dd2d5015058d8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:06.122619 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-59efe829-92a4-463b-a5fc-368af222b5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:06.219580 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a00d91-8044-4395-a2b4-1a5887026bff None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:06.230413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9da4c-52f3-4ced-94fa-fd1feb6b9ad0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:06.231867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9da4c-52f3-4ced-94fa-fd1feb6b9ad0 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:06.232290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9da4c-52f3-4ced-94fa-fd1feb6b9ad0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:06.232789 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9da4c-52f3-4ced-94fa-fd1feb6b9ad0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:06.287592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb9da4c-52f3-4ced-94fa-fd1feb6b9ad0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:06.291932 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb9da4c-52f3-4ced-94fa-fd1feb6b9ad0 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:06.345971 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 861/1733] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:05:06.359392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 873/1734] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 01:05:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:06.721290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9da4c-52f3-4ced-94fa-fd1feb6b9ad0 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:06.724446 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5bb9da4c-52f3-4ced-94fa-fd1feb6b9ad0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:07.041119 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9da4c-52f3-4ced-94fa-fd1feb6b9ad0 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:07.055511 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c35614-d6f0-49f2-9fd2-8c4399d0fbf3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e52776d9b4894af9b85a24136a8255e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:07.058339 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c35614-d6f0-49f2-9fd2-8c4399d0fbf3 service swift] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:07.058741 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c35614-d6f0-49f2-9fd2-8c4399d0fbf3 service swift] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:07.058914 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c35614-d6f0-49f2-9fd2-8c4399d0fbf3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:07.078872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c35614-d6f0-49f2-9fd2-8c4399d0fbf3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:07.082982 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c35614-d6f0-49f2-9fd2-8c4399d0fbf3 service swift] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:07.102998 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 862/1735] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 01:05:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 14 01:05:07.713725 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59efe829-92a4-463b-a5fc-368af222b5b4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:07.715537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-59efe829-92a4-463b-a5fc-368af222b5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:07.743509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59efe829-92a4-463b-a5fc-368af222b5b4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:07.749025 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b8119-8d48-4b12-afad-94b5eb8723ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:07.750098 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b8119-8d48-4b12-afad-94b5eb8723ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:07.750536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b8119-8d48-4b12-afad-94b5eb8723ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:07.750957 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b8119-8d48-4b12-afad-94b5eb8723ef admin admin] PATH_INFO: `/v3/projects/d9c3fbccc03a4770a9e6a6412cca4212` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:07.752381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9b8119-8d48-4b12-afad-94b5eb8723ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9c3fbccc03a4770a9e6a6412cca4212)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:07.754502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9b8119-8d48-4b12-afad-94b5eb8723ef admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:07.802192 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:07.817061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2b9b8119-8d48-4b12-afad-94b5eb8723ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9c3fbccc03a4770a9e6a6412cca4212', 'request_id': 'req-2b9b8119-8d48-4b12-afad-94b5eb8723ef'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:07.828268 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2b9b8119-8d48-4b12-afad-94b5eb8723ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9c3fbccc03a4770a9e6a6412cca4212', 'request_id': 'req-2b9b8119-8d48-4b12-afad-94b5eb8723ef'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:07.837675 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2b9b8119-8d48-4b12-afad-94b5eb8723ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9c3fbccc03a4770a9e6a6412cca4212', 'request_id': 'req-2b9b8119-8d48-4b12-afad-94b5eb8723ef'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:07.854139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2b9b8119-8d48-4b12-afad-94b5eb8723ef admin admin] The token cache is being invalidate because project d9c3fbccc03a4770a9e6a6412cca4212 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:07.854428 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2b9b8119-8d48-4b12-afad-94b5eb8723ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:07.855249 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2b9b8119-8d48-4b12-afad-94b5eb8723ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:07.858172 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 874/1736] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:07 2022] DELETE /identity/v3/projects/d9c3fbccc03a4770a9e6a6412cca4212 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:07.861702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c35614-d6f0-49f2-9fd2-8c4399d0fbf3 service swift] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:07.878610 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c35614-d6f0-49f2-9fd2-8c4399d0fbf3 service swift] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:07.904736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:07.971595 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d083f-d81d-453a-b109-f34f59be2610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:07.972841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d083f-d81d-453a-b109-f34f59be2610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:07.973275 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d083f-d81d-453a-b109-f34f59be2610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:07.973715 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d083f-d81d-453a-b109-f34f59be2610 admin admin] PATH_INFO: `/v3/users/7fa1e2f344b54be591e5efea7447d7e7` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:07.976628 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d083f-d81d-453a-b109-f34f59be2610 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fa1e2f344b54be591e5efea7447d7e7)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:07.978928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d083f-d81d-453a-b109-f34f59be2610 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:08.114392 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-980d083f-d81d-453a-b109-f34f59be2610 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fa1e2f344b54be591e5efea7447d7e7'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:08.115489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-980d083f-d81d-453a-b109-f34f59be2610 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fa1e2f344b54be591e5efea7447d7e7'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:08.145382 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-980d083f-d81d-453a-b109-f34f59be2610 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fa1e2f344b54be591e5efea7447d7e7'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:08.154458 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-980d083f-d81d-453a-b109-f34f59be2610 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fa1e2f344b54be591e5efea7447d7e7'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:08.157758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-980d083f-d81d-453a-b109-f34f59be2610 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fa1e2f344b54be591e5efea7447d7e7'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:08.208045 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 863/1737] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:07 2022] DELETE /identity/v3/users/7fa1e2f344b54be591e5efea7447d7e7 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:08.797024 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b8119-8d48-4b12-afad-94b5eb8723ef admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:08.799097 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2b9b8119-8d48-4b12-afad-94b5eb8723ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:08.826585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b8119-8d48-4b12-afad-94b5eb8723ef admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:08.900678 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d68866-7787-42fb-ba44-e1fcab5fbecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:08.901333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d68866-7787-42fb-ba44-e1fcab5fbecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:08.901620 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d68866-7787-42fb-ba44-e1fcab5fbecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:08.902060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d68866-7787-42fb-ba44-e1fcab5fbecc admin admin] PATH_INFO: `/v3/projects/3b088a56350e46f6bbf3bf8023b157c4` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:08.903213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d68866-7787-42fb-ba44-e1fcab5fbecc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b088a56350e46f6bbf3bf8023b157c4)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:08.905260 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d68866-7787-42fb-ba44-e1fcab5fbecc admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:08.966616 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-69d68866-7787-42fb-ba44-e1fcab5fbecc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b088a56350e46f6bbf3bf8023b157c4', 'request_id': 'req-69d68866-7787-42fb-ba44-e1fcab5fbecc'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:08.976381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-69d68866-7787-42fb-ba44-e1fcab5fbecc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b088a56350e46f6bbf3bf8023b157c4', 'request_id': 'req-69d68866-7787-42fb-ba44-e1fcab5fbecc'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:08.999324 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-69d68866-7787-42fb-ba44-e1fcab5fbecc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b088a56350e46f6bbf3bf8023b157c4', 'request_id': 'req-69d68866-7787-42fb-ba44-e1fcab5fbecc'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:09.014315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-69d68866-7787-42fb-ba44-e1fcab5fbecc admin admin] The token cache is being invalidate because project 3b088a56350e46f6bbf3bf8023b157c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:09.015170 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-69d68866-7787-42fb-ba44-e1fcab5fbecc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:09.015704 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-69d68866-7787-42fb-ba44-e1fcab5fbecc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:09.027288 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5981cd-19c9-469a-8607-cc54e51899a5 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:09.027808 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5981cd-19c9-469a-8607-cc54e51899a5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:09.028031 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5981cd-19c9-469a-8607-cc54e51899a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:09.044318 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 875/1738] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:08 2022] DELETE /identity/v3/projects/3b088a56350e46f6bbf3bf8023b157c4 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:09.069676 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-2a5981cd-19c9-469a-8607-cc54e51899a5 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:09.071328 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2a5981cd-19c9-469a-8607-cc54e51899a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:09.141806 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 864/1739] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:09.149648 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d4726-e710-4e00-945c-45d27df52ad4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:09.152203 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d4726-e710-4e00-945c-45d27df52ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:09.152621 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d4726-e710-4e00-945c-45d27df52ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:09.185502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-8a6d4726-e710-4e00-945c-45d27df52ad4 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:09.187076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8a6d4726-e710-4e00-945c-45d27df52ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:09.196761 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 876/1740] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:09.201115 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5981cd-19c9-469a-8607-cc54e51899a5 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:09.217896 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5981cd-19c9-469a-8607-cc54e51899a5 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:09.227516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0caba-a3c9-4969-906d-09f9d8219a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:09.228933 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0caba-a3c9-4969-906d-09f9d8219a44 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:09.229293 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0caba-a3c9-4969-906d-09f9d8219a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:09.231791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0caba-a3c9-4969-906d-09f9d8219a44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:09.233625 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0caba-a3c9-4969-906d-09f9d8219a44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:05:09.234104 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0caba-a3c9-4969-906d-09f9d8219a44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:09.236279 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0caba-a3c9-4969-906d-09f9d8219a44 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:09.256744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 865/1741] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:05:09.257979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d4726-e710-4e00-945c-45d27df52ad4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:09.269091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d4726-e710-4e00-945c-45d27df52ad4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:09.274917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee72cca-6a16-4185-a658-9ac2cbe41aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:09.275541 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee72cca-6a16-4185-a658-9ac2cbe41aa4 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:09.275850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee72cca-6a16-4185-a658-9ac2cbe41aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:09.276124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee72cca-6a16-4185-a658-9ac2cbe41aa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:09.276775 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee72cca-6a16-4185-a658-9ac2cbe41aa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:09.278684 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee72cca-6a16-4185-a658-9ac2cbe41aa4 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:09.312272 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 877/1742] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:05:09 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:09.316354 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0caba-a3c9-4969-906d-09f9d8219a44 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:09.328702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0caba-a3c9-4969-906d-09f9d8219a44 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:09.357881 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:09.410446 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d6af4-184e-4eb3-90ba-e9854e547821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:09.411281 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d6af4-184e-4eb3-90ba-e9854e547821 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:09.413266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d6af4-184e-4eb3-90ba-e9854e547821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:09.413667 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d6af4-184e-4eb3-90ba-e9854e547821 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:09.414636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641d6af4-184e-4eb3-90ba-e9854e547821 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:09.417518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641d6af4-184e-4eb3-90ba-e9854e547821 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:09.507513 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 866/1743] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:05:09 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:09.509345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee72cca-6a16-4185-a658-9ac2cbe41aa4 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:09.525826 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee72cca-6a16-4185-a658-9ac2cbe41aa4 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:09.532631 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b81e56-e31e-4223-9448-8b94523510a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:09.533414 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b81e56-e31e-4223-9448-8b94523510a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:09.533986 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b81e56-e31e-4223-9448-8b94523510a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:09.534398 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b81e56-e31e-4223-9448-8b94523510a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:09.535182 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b81e56-e31e-4223-9448-8b94523510a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:09.538946 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b81e56-e31e-4223-9448-8b94523510a1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:09.555870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 878/1744] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:09.559821 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d6af4-184e-4eb3-90ba-e9854e547821 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:09.573103 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d6af4-184e-4eb3-90ba-e9854e547821 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:09.582051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e56c31-8a35-4ef8-9b99-f808c96bcb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:09.582647 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e56c31-8a35-4ef8-9b99-f808c96bcb28 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:09.583029 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e56c31-8a35-4ef8-9b99-f808c96bcb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:09.583195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e56c31-8a35-4ef8-9b99-f808c96bcb28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:09.583888 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e56c31-8a35-4ef8-9b99-f808c96bcb28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:09.585607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e56c31-8a35-4ef8-9b99-f808c96bcb28 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:09.599594 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 867/1745] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:09.602806 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b81e56-e31e-4223-9448-8b94523510a1 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:09.614416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b81e56-e31e-4223-9448-8b94523510a1 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:09.619638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a394a-5e49-40a6-bc67-5123c80b5ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:09.620550 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a394a-5e49-40a6-bc67-5123c80b5ce4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:09.620914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a394a-5e49-40a6-bc67-5123c80b5ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:09.621278 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a394a-5e49-40a6-bc67-5123c80b5ce4 admin admin] PATH_INFO: `/v3/projects/11294376d60849ae8256930c6630aa24/users/18aa833ccad3454aae1fd89f6e26bbae/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:09.636930 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:09.642510 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280a394a-5e49-40a6-bc67-5123c80b5ce4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11294376d60849ae8256930c6630aa24, user_id=18aa833ccad3454aae1fd89f6e26bbae, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:09.645816 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280a394a-5e49-40a6-bc67-5123c80b5ce4 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:09.666347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 879/1746] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:09 2022] PUT /identity/v3/projects/11294376d60849ae8256930c6630aa24/users/18aa833ccad3454aae1fd89f6e26bbae/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:05:09.678039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1f4a1-3af9-42f1-8f6c-bb77bbb5e4fc None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:09.678338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1f4a1-3af9-42f1-8f6c-bb77bbb5e4fc None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:09.678615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1f4a1-3af9-42f1-8f6c-bb77bbb5e4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:09.713639 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e9f1f4a1-3af9-42f1-8f6c-bb77bbb5e4fc None None] MFA Rules not processed for user `18aa833ccad3454aae1fd89f6e26bbae`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:09.715061 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e9f1f4a1-3af9-42f1-8f6c-bb77bbb5e4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:09.809064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 868/1747] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:09.813846 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a394a-5e49-40a6-bc67-5123c80b5ce4 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:09.926319 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a394a-5e49-40a6-bc67-5123c80b5ce4 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:09.932480 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08706091-1e8e-4602-bbf2-19ec7bf0d307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:09.933579 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08706091-1e8e-4602-bbf2-19ec7bf0d307 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:09.933878 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08706091-1e8e-4602-bbf2-19ec7bf0d307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:09.934159 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08706091-1e8e-4602-bbf2-19ec7bf0d307 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:09.934823 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08706091-1e8e-4602-bbf2-19ec7bf0d307 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:09.936862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08706091-1e8e-4602-bbf2-19ec7bf0d307 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:09.985396 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 880/1748] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:05:09 2022] POST /identity/v3/projects => generated 383 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:09.987099 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1f4a1-3af9-42f1-8f6c-bb77bbb5e4fc None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:10.008596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1f4a1-3af9-42f1-8f6c-bb77bbb5e4fc None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:10.071453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581eaad-efd4-4c1e-a9bc-8e8bd8e4a40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:10.072115 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581eaad-efd4-4c1e-a9bc-8e8bd8e4a40b admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:10.072384 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581eaad-efd4-4c1e-a9bc-8e8bd8e4a40b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:10.072661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a581eaad-efd4-4c1e-a9bc-8e8bd8e4a40b admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:10.073333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a581eaad-efd4-4c1e-a9bc-8e8bd8e4a40b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:10.075270 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a581eaad-efd4-4c1e-a9bc-8e8bd8e4a40b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:10.112701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 869/1749] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:05:09 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:10.117887 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08706091-1e8e-4602-bbf2-19ec7bf0d307 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:10.128813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08706091-1e8e-4602-bbf2-19ec7bf0d307 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:10.139073 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a2860-2303-424c-bca8-9d56670ef977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:10.140076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a2860-2303-424c-bca8-9d56670ef977 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:10.140430 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a2860-2303-424c-bca8-9d56670ef977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:10.140800 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a2860-2303-424c-bca8-9d56670ef977 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:10.141554 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a2860-2303-424c-bca8-9d56670ef977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:10.143413 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a2860-2303-424c-bca8-9d56670ef977 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:10.156293 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 881/1750] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:05:10.160313 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581eaad-efd4-4c1e-a9bc-8e8bd8e4a40b admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:10.173500 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a581eaad-efd4-4c1e-a9bc-8e8bd8e4a40b admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:10.184332 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecebdc35-6e24-4d41-a6d3-1c5a4918a212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:10.185230 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecebdc35-6e24-4d41-a6d3-1c5a4918a212 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:10.185676 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecebdc35-6e24-4d41-a6d3-1c5a4918a212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:10.186167 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecebdc35-6e24-4d41-a6d3-1c5a4918a212 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:10.186971 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecebdc35-6e24-4d41-a6d3-1c5a4918a212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:10.190573 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecebdc35-6e24-4d41-a6d3-1c5a4918a212 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:10.211452 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 870/1751] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:10.212823 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a2860-2303-424c-bca8-9d56670ef977 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:10.235703 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a2860-2303-424c-bca8-9d56670ef977 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:10.246657 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aeec66-e3d3-484d-8748-f6d2fb8d01e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:10.247767 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aeec66-e3d3-484d-8748-f6d2fb8d01e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:10.248314 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aeec66-e3d3-484d-8748-f6d2fb8d01e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:10.248740 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aeec66-e3d3-484d-8748-f6d2fb8d01e9 admin admin] PATH_INFO: `/v3/projects/73aa6c4496a74ce3a59ee8846f8f4364/users/1564db0754a54c7caba26274574d823e/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:10.282188 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aeec66-e3d3-484d-8748-f6d2fb8d01e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73aa6c4496a74ce3a59ee8846f8f4364, user_id=1564db0754a54c7caba26274574d823e, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:10.286076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aeec66-e3d3-484d-8748-f6d2fb8d01e9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:10.346720 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 882/1752] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:10 2022] PUT /identity/v3/projects/73aa6c4496a74ce3a59ee8846f8f4364/users/1564db0754a54c7caba26274574d823e/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:10.359607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04e945-9d82-441c-ad04-2c2ddad4a55b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:10.360098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04e945-9d82-441c-ad04-2c2ddad4a55b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:10.360619 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04e945-9d82-441c-ad04-2c2ddad4a55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:10.394611 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:10.409798 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-2f04e945-9d82-441c-ad04-2c2ddad4a55b None None] MFA Rules not processed for user `1564db0754a54c7caba26274574d823e`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:10.411033 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2f04e945-9d82-441c-ad04-2c2ddad4a55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:10.535634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 871/1753] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:10.541118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aeec66-e3d3-484d-8748-f6d2fb8d01e9 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:10.759264 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aeec66-e3d3-484d-8748-f6d2fb8d01e9 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:10.788371 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5646dd7a-2400-4ef4-b50b-96507dddcd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:10.789433 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646dd7a-2400-4ef4-b50b-96507dddcd86 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:10.792982 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646dd7a-2400-4ef4-b50b-96507dddcd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:10.797135 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646dd7a-2400-4ef4-b50b-96507dddcd86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:10.798098 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5646dd7a-2400-4ef4-b50b-96507dddcd86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:10.800220 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5646dd7a-2400-4ef4-b50b-96507dddcd86 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:10.873952 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab2776-21f0-4484-8900-3ecd04bc1622 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:10.874548 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab2776-21f0-4484-8900-3ecd04bc1622 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:10.874997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab2776-21f0-4484-8900-3ecd04bc1622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:10.979152 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-52ab2776-21f0-4484-8900-3ecd04bc1622 None None] MFA Rules not processed for user `4d64f9bca7c546e4ba7f54c1c47d665e`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:10.994597 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-52ab2776-21f0-4484-8900-3ecd04bc1622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:11.047983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 883/1754] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:05:10 2022] POST /identity/v3/projects => generated 385 bytes in 507 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:11.053492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5646dd7a-2400-4ef4-b50b-96507dddcd86 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:11.166354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:11.364676 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 872/1755] 10.222.0.44 () {58 vars in 1015 bytes} [Wed Sep 14 01:05:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 491 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:11.423754 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5646dd7a-2400-4ef4-b50b-96507dddcd86 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:11.441479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4d4e3-a6b4-492c-ac5a-eaadfa39cc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:11.442285 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4d4e3-a6b4-492c-ac5a-eaadfa39cc86 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:11.442633 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4d4e3-a6b4-492c-ac5a-eaadfa39cc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:11.442915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4d4e3-a6b4-492c-ac5a-eaadfa39cc86 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:11.444688 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e4d4e3-a6b4-492c-ac5a-eaadfa39cc86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:11.446727 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e4d4e3-a6b4-492c-ac5a-eaadfa39cc86 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:11.450003 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab2776-21f0-4484-8900-3ecd04bc1622 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:11.451650 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-52ab2776-21f0-4484-8900-3ecd04bc1622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:11.474609 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab2776-21f0-4484-8900-3ecd04bc1622 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:11.567625 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 884/1756] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:05:11 2022] POST /identity/v3/users => generated 299 bytes in 512 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:11.572250 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4d4e3-a6b4-492c-ac5a-eaadfa39cc86 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:11.678751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088dcd7-5e02-4502-a188-85698e473d4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:11.679622 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088dcd7-5e02-4502-a188-85698e473d4f service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:11.679915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088dcd7-5e02-4502-a188-85698e473d4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:11.680194 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088dcd7-5e02-4502-a188-85698e473d4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:11.699871 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088dcd7-5e02-4502-a188-85698e473d4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:11.701366 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088dcd7-5e02-4502-a188-85698e473d4f service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:11.740796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 873/1757] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:05:11.741942 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088dcd7-5e02-4502-a188-85698e473d4f service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:11.747583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4d4e3-a6b4-492c-ac5a-eaadfa39cc86 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:11.761582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c458c-3f25-4d14-8dca-5573011cf73d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:11.772889 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c458c-3f25-4d14-8dca-5573011cf73d service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:11.773191 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c458c-3f25-4d14-8dca-5573011cf73d service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:11.773470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c458c-3f25-4d14-8dca-5573011cf73d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:11.809498 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088dcd7-5e02-4502-a188-85698e473d4f service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:11.819158 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c458c-3f25-4d14-8dca-5573011cf73d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:11.830468 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c458c-3f25-4d14-8dca-5573011cf73d service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:11.842552 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c6a67-ec9b-4df2-a0de-5ee9bedd709c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:11.843193 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c6a67-ec9b-4df2-a0de-5ee9bedd709c admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:11.865906 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c6a67-ec9b-4df2-a0de-5ee9bedd709c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:11.866285 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c6a67-ec9b-4df2-a0de-5ee9bedd709c admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:11.866999 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c6a67-ec9b-4df2-a0de-5ee9bedd709c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:11.875540 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c6a67-ec9b-4df2-a0de-5ee9bedd709c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:11.899273 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 885/1758] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:11.932445 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c458c-3f25-4d14-8dca-5573011cf73d service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:11.941094 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 874/1759] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:11.957379 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c458c-3f25-4d14-8dca-5573011cf73d service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:11.970736 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb99af-ff86-484a-a8c8-9c7179a3badf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:11.971523 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb99af-ff86-484a-a8c8-9c7179a3badf admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:11.972131 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb99af-ff86-484a-a8c8-9c7179a3badf admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:11.972487 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb99af-ff86-484a-a8c8-9c7179a3badf admin admin] PATH_INFO: `/v3/projects/733ece650f4f4b50bb2bc6f155d0386c/users/7828dfc837724c3dada86fa1303ea48c/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:12.023506 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eb99af-ff86-484a-a8c8-9c7179a3badf admin admin] RBAC: Authorizing `identity:create_grant(project_id=733ece650f4f4b50bb2bc6f155d0386c, user_id=7828dfc837724c3dada86fa1303ea48c, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:12.029101 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eb99af-ff86-484a-a8c8-9c7179a3badf admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:12.114871 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 886/1760] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:11 2022] PUT /identity/v3/projects/733ece650f4f4b50bb2bc6f155d0386c/users/7828dfc837724c3dada86fa1303ea48c/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:12.116937 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c6a67-ec9b-4df2-a0de-5ee9bedd709c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:12.132932 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c6a67-ec9b-4df2-a0de-5ee9bedd709c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:12.143293 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb99af-ff86-484a-a8c8-9c7179a3badf admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:12.145578 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-73eb99af-ff86-484a-a8c8-9c7179a3badf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:12.314982 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:12.367828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3bc6d-f935-4518-98c8-8b2f07342d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:12.369224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3bc6d-f935-4518-98c8-8b2f07342d74 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:12.369696 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3bc6d-f935-4518-98c8-8b2f07342d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:12.370139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3bc6d-f935-4518-98c8-8b2f07342d74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:12.370947 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3bc6d-f935-4518-98c8-8b2f07342d74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:12.372809 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3bc6d-f935-4518-98c8-8b2f07342d74 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:12.394339 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 875/1761] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:05:12.418325 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3bc6d-f935-4518-98c8-8b2f07342d74 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:12.438538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3bc6d-f935-4518-98c8-8b2f07342d74 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:12.486316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bba5c4-69ca-4709-a250-cb946baf58b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:12.487019 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bba5c4-69ca-4709-a250-cb946baf58b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:12.487299 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bba5c4-69ca-4709-a250-cb946baf58b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:12.487591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bba5c4-69ca-4709-a250-cb946baf58b7 admin admin] PATH_INFO: `/v3/projects/733ece650f4f4b50bb2bc6f155d0386c/users/7828dfc837724c3dada86fa1303ea48c/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:12.496961 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bba5c4-69ca-4709-a250-cb946baf58b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=733ece650f4f4b50bb2bc6f155d0386c, user_id=7828dfc837724c3dada86fa1303ea48c, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:12.499859 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bba5c4-69ca-4709-a250-cb946baf58b7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:12.534447 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eb99af-ff86-484a-a8c8-9c7179a3badf admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:12.540582 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 876/1762] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:12 2022] PUT /identity/v3/projects/733ece650f4f4b50bb2bc6f155d0386c/users/7828dfc837724c3dada86fa1303ea48c/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:12.542414 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ae552-8092-46c5-82ae-d99593b838f9 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:12.542872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ae552-8092-46c5-82ae-d99593b838f9 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:12.543300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ae552-8092-46c5-82ae-d99593b838f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:12.648612 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-1f6ae552-8092-46c5-82ae-d99593b838f9 None None] MFA Rules not processed for user `7828dfc837724c3dada86fa1303ea48c`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:12.654033 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1f6ae552-8092-46c5-82ae-d99593b838f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:12.859339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:12.955036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b807e4-2d78-474e-8ee5-bb73fbdb60f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:12.962943 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b807e4-2d78-474e-8ee5-bb73fbdb60f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:12.963733 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b807e4-2d78-474e-8ee5-bb73fbdb60f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:12.964241 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b807e4-2d78-474e-8ee5-bb73fbdb60f9 admin admin] PATH_INFO: `/v3/users/49c0500eb9c84f3595cb1558553fd378` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:12.967699 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b807e4-2d78-474e-8ee5-bb73fbdb60f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49c0500eb9c84f3595cb1558553fd378)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:12.981538 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b807e4-2d78-474e-8ee5-bb73fbdb60f9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:13.030379 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 877/1763] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:12 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 489 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:13.039894 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ae552-8092-46c5-82ae-d99593b838f9 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:13.044077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1f6ae552-8092-46c5-82ae-d99593b838f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:13.119220 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-57b807e4-2d78-474e-8ee5-bb73fbdb60f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49c0500eb9c84f3595cb1558553fd378'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:13.120096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-57b807e4-2d78-474e-8ee5-bb73fbdb60f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49c0500eb9c84f3595cb1558553fd378'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:13.136812 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-57b807e4-2d78-474e-8ee5-bb73fbdb60f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49c0500eb9c84f3595cb1558553fd378'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:13.137645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-57b807e4-2d78-474e-8ee5-bb73fbdb60f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49c0500eb9c84f3595cb1558553fd378'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:13.153930 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-57b807e4-2d78-474e-8ee5-bb73fbdb60f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49c0500eb9c84f3595cb1558553fd378'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:13.244751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 887/1764] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:12 2022] DELETE /identity/v3/users/49c0500eb9c84f3595cb1558553fd378 => generated 0 bytes in 1098 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:05:13.248916 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05f274-060c-4d1a-93af-5213be544075 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:13.251067 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05f274-060c-4d1a-93af-5213be544075 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:13.251481 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05f274-060c-4d1a-93af-5213be544075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:13.295969 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-6c05f274-060c-4d1a-93af-5213be544075 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:13.298979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6c05f274-060c-4d1a-93af-5213be544075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:13.355159 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ae552-8092-46c5-82ae-d99593b838f9 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:13.546612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 888/1765] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:13.592834 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf2b3d-24b6-4a78-955e-acbb23db8f69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:13.594236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf2b3d-24b6-4a78-955e-acbb23db8f69 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:13.594674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf2b3d-24b6-4a78-955e-acbb23db8f69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:13.595305 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf2b3d-24b6-4a78-955e-acbb23db8f69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:13.598862 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-cabf2b3d-24b6-4a78-955e-acbb23db8f69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:13.670412 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:13.699320 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05f274-060c-4d1a-93af-5213be544075 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:13.703158 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6c05f274-060c-4d1a-93af-5213be544075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:13.826142 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf2b3d-24b6-4a78-955e-acbb23db8f69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:13.827710 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf2b3d-24b6-4a78-955e-acbb23db8f69 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:13.852928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 878/1766] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 813 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:13.893139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05f274-060c-4d1a-93af-5213be544075 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:13.904561 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913e700-a568-42d3-beb2-310f58283612 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:13.905385 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913e700-a568-42d3-beb2-310f58283612 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:13.905783 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913e700-a568-42d3-beb2-310f58283612 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:13.906359 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913e700-a568-42d3-beb2-310f58283612 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:13.928554 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3913e700-a568-42d3-beb2-310f58283612 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:13.932346 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3913e700-a568-42d3-beb2-310f58283612 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:13.957050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 889/1767] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:15.434260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf2b3d-24b6-4a78-955e-acbb23db8f69 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:15.436467 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-cabf2b3d-24b6-4a78-955e-acbb23db8f69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:15.456920 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf2b3d-24b6-4a78-955e-acbb23db8f69 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:15.471202 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094657a7-0874-4d71-ab0a-1844a7f372f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:15.472276 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094657a7-0874-4d71-ab0a-1844a7f372f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:15.472792 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094657a7-0874-4d71-ab0a-1844a7f372f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:15.473279 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094657a7-0874-4d71-ab0a-1844a7f372f0 admin admin] PATH_INFO: `/v3/users/4c42d88828b441f1b0827f5b71f2f6da` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:15.476658 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094657a7-0874-4d71-ab0a-1844a7f372f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c42d88828b441f1b0827f5b71f2f6da)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:15.488675 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094657a7-0874-4d71-ab0a-1844a7f372f0 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:15.647181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913e700-a568-42d3-beb2-310f58283612 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:15.649978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3913e700-a568-42d3-beb2-310f58283612 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:15.665553 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-094657a7-0874-4d71-ab0a-1844a7f372f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c42d88828b441f1b0827f5b71f2f6da'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:15.668162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:15.669627 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-094657a7-0874-4d71-ab0a-1844a7f372f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c42d88828b441f1b0827f5b71f2f6da'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:15.689895 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913e700-a568-42d3-beb2-310f58283612 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:15.696940 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-094657a7-0874-4d71-ab0a-1844a7f372f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c42d88828b441f1b0827f5b71f2f6da'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:15.709078 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-094657a7-0874-4d71-ab0a-1844a7f372f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c42d88828b441f1b0827f5b71f2f6da'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:15.710095 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce2aff-0d5f-4b9e-9e7d-0683fd786973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:15.711009 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce2aff-0d5f-4b9e-9e7d-0683fd786973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:15.711423 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce2aff-0d5f-4b9e-9e7d-0683fd786973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:15.711744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-094657a7-0874-4d71-ab0a-1844a7f372f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c42d88828b441f1b0827f5b71f2f6da'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:15.711977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce2aff-0d5f-4b9e-9e7d-0683fd786973 admin admin] PATH_INFO: `/v3/projects/9406307e226f40e7953ab4dad717bae5` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:15.713392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce2aff-0d5f-4b9e-9e7d-0683fd786973 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9406307e226f40e7953ab4dad717bae5)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:15.715827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce2aff-0d5f-4b9e-9e7d-0683fd786973 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:15.776866 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 879/1768] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:15 2022] DELETE /identity/v3/users/4c42d88828b441f1b0827f5b71f2f6da => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:15.781324 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446a7bc-f24f-4a52-b5c1-6175acbae9be None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:15.781611 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446a7bc-f24f-4a52-b5c1-6175acbae9be None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:15.782574 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446a7bc-f24f-4a52-b5c1-6175acbae9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:15.822968 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9fce2aff-0d5f-4b9e-9e7d-0683fd786973 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9406307e226f40e7953ab4dad717bae5', 'request_id': 'req-9fce2aff-0d5f-4b9e-9e7d-0683fd786973'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:15.840636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9fce2aff-0d5f-4b9e-9e7d-0683fd786973 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9406307e226f40e7953ab4dad717bae5', 'request_id': 'req-9fce2aff-0d5f-4b9e-9e7d-0683fd786973'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:15.857892 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9fce2aff-0d5f-4b9e-9e7d-0683fd786973 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9406307e226f40e7953ab4dad717bae5', 'request_id': 'req-9fce2aff-0d5f-4b9e-9e7d-0683fd786973'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:15.866025 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-0446a7bc-f24f-4a52-b5c1-6175acbae9be None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:15.867298 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0446a7bc-f24f-4a52-b5c1-6175acbae9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:15.880637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9fce2aff-0d5f-4b9e-9e7d-0683fd786973 admin admin] The token cache is being invalidate because project 9406307e226f40e7953ab4dad717bae5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:15.881702 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9fce2aff-0d5f-4b9e-9e7d-0683fd786973 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:15.892088 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-9fce2aff-0d5f-4b9e-9e7d-0683fd786973 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:15.894497 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 890/1769] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:15 2022] DELETE /identity/v3/projects/9406307e226f40e7953ab4dad717bae5 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:15.898819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce2aff-0d5f-4b9e-9e7d-0683fd786973 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:15.901133 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9fce2aff-0d5f-4b9e-9e7d-0683fd786973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:16.003758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 880/1770] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:16.093687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446a7bc-f24f-4a52-b5c1-6175acbae9be None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:16.102894 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0446a7bc-f24f-4a52-b5c1-6175acbae9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:16.124468 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce2aff-0d5f-4b9e-9e7d-0683fd786973 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:16.140406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371dbd0-ac08-493f-8864-5112a443327a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:16.141417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371dbd0-ac08-493f-8864-5112a443327a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:16.141923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371dbd0-ac08-493f-8864-5112a443327a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:16.142478 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371dbd0-ac08-493f-8864-5112a443327a admin admin] PATH_INFO: `/v3/users/19e6ff6be5034efeb7f56994371f4813` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:16.144274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371dbd0-ac08-493f-8864-5112a443327a admin admin] RBAC: Authorizing `identity:delete_user(user_id=19e6ff6be5034efeb7f56994371f4813)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:16.147027 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371dbd0-ac08-493f-8864-5112a443327a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:16.227628 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:16.256379 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e371dbd0-ac08-493f-8864-5112a443327a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19e6ff6be5034efeb7f56994371f4813'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:16.264055 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e371dbd0-ac08-493f-8864-5112a443327a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19e6ff6be5034efeb7f56994371f4813'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:16.277502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e371dbd0-ac08-493f-8864-5112a443327a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19e6ff6be5034efeb7f56994371f4813'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:16.278768 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e371dbd0-ac08-493f-8864-5112a443327a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19e6ff6be5034efeb7f56994371f4813'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:16.289890 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e371dbd0-ac08-493f-8864-5112a443327a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19e6ff6be5034efeb7f56994371f4813'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:16.313534 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446a7bc-f24f-4a52-b5c1-6175acbae9be None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:16.326044 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6277e-2c9c-45f7-aa61-e4da096692a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:16.326714 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6277e-2c9c-45f7-aa61-e4da096692a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:16.326999 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6277e-2c9c-45f7-aa61-e4da096692a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:16.327275 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6277e-2c9c-45f7-aa61-e4da096692a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:16.329515 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-13a6277e-2c9c-45f7-aa61-e4da096692a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:16.346246 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 891/1771] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:15 2022] DELETE /identity/v3/users/19e6ff6be5034efeb7f56994371f4813 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:16.355951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6277e-2c9c-45f7-aa61-e4da096692a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:16.357430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6277e-2c9c-45f7-aa61-e4da096692a7 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:16.398514 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371dbd0-ac08-493f-8864-5112a443327a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:16.400523 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e371dbd0-ac08-493f-8864-5112a443327a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:16.401938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 881/1772] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:16.408704 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:16.619349 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371dbd0-ac08-493f-8864-5112a443327a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:16.646274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af190e4f-6c76-4050-a531-ab620f1b52c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:16.647978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af190e4f-6c76-4050-a531-ab620f1b52c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:16.649540 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af190e4f-6c76-4050-a531-ab620f1b52c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:16.650038 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af190e4f-6c76-4050-a531-ab620f1b52c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:16.654404 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-af190e4f-6c76-4050-a531-ab620f1b52c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:16.814480 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af190e4f-6c76-4050-a531-ab620f1b52c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:16.816279 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af190e4f-6c76-4050-a531-ab620f1b52c1 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:16.832031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 892/1773] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:17.780895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca195f-c551-49ef-8105-805eaae5c8bf None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:17.786338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca195f-c551-49ef-8105-805eaae5c8bf None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:17.786792 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca195f-c551-49ef-8105-805eaae5c8bf None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:17.790418 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 882/1774] 10.222.0.42 () {54 vars in 921 bytes} [Wed Sep 14 01:05:17 2022] GET /identity => generated 270 bytes in 32 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:05:18.113184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb60a4f-d9e3-47d6-84e6-afec2f3dbf4b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:18.116742 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb60a4f-d9e3-47d6-84e6-afec2f3dbf4b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:18.117100 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb60a4f-d9e3-47d6-84e6-afec2f3dbf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:18.223352 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-acb60a4f-d9e3-47d6-84e6-afec2f3dbf4b None None] MFA Rules not processed for user `5e0fe2e84f6748c992c9beedf2f5d49b`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:18.231460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-acb60a4f-d9e3-47d6-84e6-afec2f3dbf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:18.407326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca195f-c551-49ef-8105-805eaae5c8bf None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:18.440233 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca195f-c551-49ef-8105-805eaae5c8bf None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:18.459425 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faca2c3-0e1b-4b14-9ced-18d2613a9e09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:18.461511 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faca2c3-0e1b-4b14-9ced-18d2613a9e09 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:18.463932 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faca2c3-0e1b-4b14-9ced-18d2613a9e09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:18.464881 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faca2c3-0e1b-4b14-9ced-18d2613a9e09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:18.480435 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4faca2c3-0e1b-4b14-9ced-18d2613a9e09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:18.504639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 893/1775] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Sep 14 01:05:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:18.735521 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faca2c3-0e1b-4b14-9ced-18d2613a9e09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:18.737426 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faca2c3-0e1b-4b14-9ced-18d2613a9e09 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:18.753140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 883/1776] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:18 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:05:19.183767 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb60a4f-d9e3-47d6-84e6-afec2f3dbf4b None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:19.186622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-acb60a4f-d9e3-47d6-84e6-afec2f3dbf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:19.221780 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb60a4f-d9e3-47d6-84e6-afec2f3dbf4b None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:19.243367 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169b603-8547-4ef7-881f-1844c2d1dd3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:19.244069 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169b603-8547-4ef7-881f-1844c2d1dd3b service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:19.244357 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169b603-8547-4ef7-881f-1844c2d1dd3b service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:19.244635 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169b603-8547-4ef7-881f-1844c2d1dd3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:19.274254 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d169b603-8547-4ef7-881f-1844c2d1dd3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:19.285623 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d169b603-8547-4ef7-881f-1844c2d1dd3b service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:19.310027 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 894/1777] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:20.208194 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faca2c3-0e1b-4b14-9ced-18d2613a9e09 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:20.212733 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4faca2c3-0e1b-4b14-9ced-18d2613a9e09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:20.382868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faca2c3-0e1b-4b14-9ced-18d2613a9e09 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:20.393221 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169b603-8547-4ef7-881f-1844c2d1dd3b service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:20.395288 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d169b603-8547-4ef7-881f-1844c2d1dd3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:20.401181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c68d2c-4ed4-498e-85e2-a9d5a679ade8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:20.401893 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c68d2c-4ed4-498e-85e2-a9d5a679ade8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:20.402195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c68d2c-4ed4-498e-85e2-a9d5a679ade8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:20.402469 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c68d2c-4ed4-498e-85e2-a9d5a679ade8 admin admin] PATH_INFO: `/v3/projects/657342491db0483a98707ab349b329ad` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:20.404867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c68d2c-4ed4-498e-85e2-a9d5a679ade8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=657342491db0483a98707ab349b329ad)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:20.406841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c68d2c-4ed4-498e-85e2-a9d5a679ade8 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:20.427182 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169b603-8547-4ef7-881f-1844c2d1dd3b service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:20.441243 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5866b3-f8c6-425d-8573-3578bb570207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:20.442787 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5866b3-f8c6-425d-8573-3578bb570207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:20.444303 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5866b3-f8c6-425d-8573-3578bb570207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:20.446437 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5866b3-f8c6-425d-8573-3578bb570207 admin admin] PATH_INFO: `/v3/projects/276429f3f3ba47438a625cc048752e5d` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:20.448439 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5866b3-f8c6-425d-8573-3578bb570207 admin admin] RBAC: Authorizing `identity:delete_project(project_id=276429f3f3ba47438a625cc048752e5d)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:20.461154 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5866b3-f8c6-425d-8573-3578bb570207 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:20.521997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-69c68d2c-4ed4-498e-85e2-a9d5a679ade8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '657342491db0483a98707ab349b329ad', 'request_id': 'req-69c68d2c-4ed4-498e-85e2-a9d5a679ade8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:20.539854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-69c68d2c-4ed4-498e-85e2-a9d5a679ade8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '657342491db0483a98707ab349b329ad', 'request_id': 'req-69c68d2c-4ed4-498e-85e2-a9d5a679ade8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:20.564181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-69c68d2c-4ed4-498e-85e2-a9d5a679ade8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '657342491db0483a98707ab349b329ad', 'request_id': 'req-69c68d2c-4ed4-498e-85e2-a9d5a679ade8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:20.576862 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:20.582030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-69c68d2c-4ed4-498e-85e2-a9d5a679ade8 admin admin] The token cache is being invalidate because project 657342491db0483a98707ab349b329ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:20.583127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-69c68d2c-4ed4-498e-85e2-a9d5a679ade8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:20.583930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-69c68d2c-4ed4-498e-85e2-a9d5a679ade8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:20.588994 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 884/1778] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:20 2022] DELETE /identity/v3/projects/657342491db0483a98707ab349b329ad => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:20.590424 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2b773-c91d-437e-b9a6-40e712c04133 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:20.590727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2b773-c91d-437e-b9a6-40e712c04133 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:20.591014 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2b773-c91d-437e-b9a6-40e712c04133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:20.622602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:20.636309 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-27f2b773-c91d-437e-b9a6-40e712c04133 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:20.637557 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-27f2b773-c91d-437e-b9a6-40e712c04133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:20.692856 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ea5866b3-f8c6-425d-8573-3578bb570207 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '276429f3f3ba47438a625cc048752e5d', 'request_id': 'req-ea5866b3-f8c6-425d-8573-3578bb570207'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:20.723874 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ea5866b3-f8c6-425d-8573-3578bb570207 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '276429f3f3ba47438a625cc048752e5d', 'request_id': 'req-ea5866b3-f8c6-425d-8573-3578bb570207'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:20.757414 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ea5866b3-f8c6-425d-8573-3578bb570207 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '276429f3f3ba47438a625cc048752e5d', 'request_id': 'req-ea5866b3-f8c6-425d-8573-3578bb570207'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:20.827082 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ea5866b3-f8c6-425d-8573-3578bb570207 admin admin] The token cache is being invalidate because project 276429f3f3ba47438a625cc048752e5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:20.829688 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ea5866b3-f8c6-425d-8573-3578bb570207 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:20.838290 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ea5866b3-f8c6-425d-8573-3578bb570207 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:20.856203 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 895/1779] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:20 2022] DELETE /identity/v3/projects/276429f3f3ba47438a625cc048752e5d => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:20.860900 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f50c55-8ec8-48bb-bba0-04429aab5dfa None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:20.862007 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f50c55-8ec8-48bb-bba0-04429aab5dfa None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:20.862290 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f50c55-8ec8-48bb-bba0-04429aab5dfa None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:20.863710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 896/1780] 10.222.0.42 () {54 vars in 921 bytes} [Wed Sep 14 01:05:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:05:20.864471 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f50c55-8ec8-48bb-bba0-04429aab5dfa None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:20.876400 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f8f50c55-8ec8-48bb-bba0-04429aab5dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:20.886912 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 885/1781] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:20.889120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee2d72-11e4-4d13-8ad2-4ec851053197 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:20.889684 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee2d72-11e4-4d13-8ad2-4ec851053197 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:20.890702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee2d72-11e4-4d13-8ad2-4ec851053197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:20.945698 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f50c55-8ec8-48bb-bba0-04429aab5dfa None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:20.968465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-31ee2d72-11e4-4d13-8ad2-4ec851053197 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:20.970643 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a4b27-3b22-4883-856c-e85e8494923c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:20.971312 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a4b27-3b22-4883-856c-e85e8494923c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:20.971778 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a4b27-3b22-4883-856c-e85e8494923c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:20.972067 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a4b27-3b22-4883-856c-e85e8494923c admin admin] PATH_INFO: `/v3/users/763efea976a544869a371ebfadb7f7ad` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:20.973421 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-31ee2d72-11e4-4d13-8ad2-4ec851053197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:20.974766 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a4b27-3b22-4883-856c-e85e8494923c admin admin] RBAC: Authorizing `identity:delete_user(user_id=763efea976a544869a371ebfadb7f7ad)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:20.977144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a4b27-3b22-4883-856c-e85e8494923c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:20.999750 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 886/1782] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:05:21.001073 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee2d72-11e4-4d13-8ad2-4ec851053197 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:21.024647 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee2d72-11e4-4d13-8ad2-4ec851053197 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:21.044310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2d00b-c21b-4d7a-a945-55ee75b5ea5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:21.045793 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2d00b-c21b-4d7a-a945-55ee75b5ea5d admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:21.047547 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2d00b-c21b-4d7a-a945-55ee75b5ea5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:21.048112 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2d00b-c21b-4d7a-a945-55ee75b5ea5d admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:21.049015 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2d00b-c21b-4d7a-a945-55ee75b5ea5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:05:21.049612 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2d00b-c21b-4d7a-a945-55ee75b5ea5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:21.055208 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2d00b-c21b-4d7a-a945-55ee75b5ea5d admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:21.095745 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 887/1783] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:05:21.097107 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2d00b-c21b-4d7a-a945-55ee75b5ea5d admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:21.113956 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2d00b-c21b-4d7a-a945-55ee75b5ea5d admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:21.115905 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e50a4b27-3b22-4883-856c-e85e8494923c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '763efea976a544869a371ebfadb7f7ad'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:21.117055 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e50a4b27-3b22-4883-856c-e85e8494923c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '763efea976a544869a371ebfadb7f7ad'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:21.123909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a595c5-1b54-457a-a53c-28fc93ec4d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:21.135155 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e50a4b27-3b22-4883-856c-e85e8494923c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '763efea976a544869a371ebfadb7f7ad'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:21.138417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e50a4b27-3b22-4883-856c-e85e8494923c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '763efea976a544869a371ebfadb7f7ad'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:21.138795 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a595c5-1b54-457a-a53c-28fc93ec4d93 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:21.139368 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a595c5-1b54-457a-a53c-28fc93ec4d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:21.140159 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a595c5-1b54-457a-a53c-28fc93ec4d93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:21.141874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a595c5-1b54-457a-a53c-28fc93ec4d93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:21.147835 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a595c5-1b54-457a-a53c-28fc93ec4d93 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:21.164775 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e50a4b27-3b22-4883-856c-e85e8494923c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '763efea976a544869a371ebfadb7f7ad'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:21.216035 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 888/1784] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:05:21 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:21.218201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecc93b-1462-4675-a89d-8793ccf39ef1 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:21.218732 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecc93b-1462-4675-a89d-8793ccf39ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:21.219242 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecc93b-1462-4675-a89d-8793ccf39ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:21.247665 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 897/1785] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:20 2022] DELETE /identity/v3/users/763efea976a544869a371ebfadb7f7ad => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:21.252277 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a4b27-3b22-4883-856c-e85e8494923c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:21.254931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e50a4b27-3b22-4883-856c-e85e8494923c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:21.270375 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-4fecc93b-1462-4675-a89d-8793ccf39ef1 None None] MFA Rules not processed for user `5e0fe2e84f6748c992c9beedf2f5d49b`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:21.272640 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4fecc93b-1462-4675-a89d-8793ccf39ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:21.399480 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 889/1786] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Sep 14 01:05:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:21.538069 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a4b27-3b22-4883-856c-e85e8494923c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:21.554648 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8890f1-1553-4e1c-bc82-f27c0eb463d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:21.555471 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8890f1-1553-4e1c-bc82-f27c0eb463d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:21.557711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8890f1-1553-4e1c-bc82-f27c0eb463d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:21.558172 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8890f1-1553-4e1c-bc82-f27c0eb463d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:21.559182 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8890f1-1553-4e1c-bc82-f27c0eb463d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:21.566223 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8890f1-1553-4e1c-bc82-f27c0eb463d8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:21.619585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 898/1787] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:05:21 2022] POST /identity/v3/users => generated 289 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:21.624379 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fecc93b-1462-4675-a89d-8793ccf39ef1 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:21.641175 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8890f1-1553-4e1c-bc82-f27c0eb463d8 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:21.651744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fecc93b-1462-4675-a89d-8793ccf39ef1 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:21.652886 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8e8890f1-1553-4e1c-bc82-f27c0eb463d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:21.673789 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46944730-a731-4562-9bbe-ef0311bd43f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:21.676947 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46944730-a731-4562-9bbe-ef0311bd43f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:21.680337 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46944730-a731-4562-9bbe-ef0311bd43f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:21.680490 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46944730-a731-4562-9bbe-ef0311bd43f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:21.681309 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46944730-a731-4562-9bbe-ef0311bd43f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:21.689142 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46944730-a731-4562-9bbe-ef0311bd43f8 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:21.721775 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8890f1-1553-4e1c-bc82-f27c0eb463d8 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:21.729146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 890/1788] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:21.735326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46944730-a731-4562-9bbe-ef0311bd43f8 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:21.751686 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f52e2-cd46-41ea-9e6d-a7469bb5f600 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:21.752837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f52e2-cd46-41ea-9e6d-a7469bb5f600 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:21.754419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f52e2-cd46-41ea-9e6d-a7469bb5f600 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:21.754812 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f52e2-cd46-41ea-9e6d-a7469bb5f600 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:21.778395 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46944730-a731-4562-9bbe-ef0311bd43f8 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:21.785966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f52e2-cd46-41ea-9e6d-a7469bb5f600 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:21.789930 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f52e2-cd46-41ea-9e6d-a7469bb5f600 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:21.799351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e1cbd-aedd-4fa8-a00f-3d9895841538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:21.800321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e1cbd-aedd-4fa8-a00f-3d9895841538 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:21.803046 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e1cbd-aedd-4fa8-a00f-3d9895841538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:21.803454 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e1cbd-aedd-4fa8-a00f-3d9895841538 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:21.804441 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e1cbd-aedd-4fa8-a00f-3d9895841538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:21.813896 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e1cbd-aedd-4fa8-a00f-3d9895841538 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:21.849602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 899/1789] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:21.859963 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f52e2-cd46-41ea-9e6d-a7469bb5f600 service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:21.865893 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 891/1790] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:21.883674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f52e2-cd46-41ea-9e6d-a7469bb5f600 service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:21.901899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698b113-d2ab-4037-b49a-b735a93aa01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:21.902669 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698b113-d2ab-4037-b49a-b735a93aa01d admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:21.903026 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698b113-d2ab-4037-b49a-b735a93aa01d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:21.903382 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698b113-d2ab-4037-b49a-b735a93aa01d admin admin] PATH_INFO: `/v3/projects/4a3bca454f1f4946a2edf357b21530a2/users/1f7b68b9544a416c832526f69d591a8d/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:21.908790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=80,keystone:DELETE=15,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:21.942077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9698b113-d2ab-4037-b49a-b735a93aa01d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a3bca454f1f4946a2edf357b21530a2, user_id=1f7b68b9544a416c832526f69d591a8d, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:21.949184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9698b113-d2ab-4037-b49a-b735a93aa01d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:21.977290 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 900/1791] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:21 2022] PUT /identity/v3/projects/4a3bca454f1f4946a2edf357b21530a2/users/1f7b68b9544a416c832526f69d591a8d/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:21.994258 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec589f6-5928-4cd4-a961-6fd9a50bdab4 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:21.994555 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec589f6-5928-4cd4-a961-6fd9a50bdab4 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:21.994832 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec589f6-5928-4cd4-a961-6fd9a50bdab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:22.054857 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-aec589f6-5928-4cd4-a961-6fd9a50bdab4 None None] MFA Rules not processed for user `1f7b68b9544a416c832526f69d591a8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:22.056459 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-aec589f6-5928-4cd4-a961-6fd9a50bdab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:22.099926 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:22.232440 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 892/1792] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:22.236966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698b113-d2ab-4037-b49a-b735a93aa01d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:22.388971 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698b113-d2ab-4037-b49a-b735a93aa01d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:22.408602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f858a4ac-fe83-41ae-bb83-99f67bc8f0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:22.409432 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f858a4ac-fe83-41ae-bb83-99f67bc8f0e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:22.409756 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f858a4ac-fe83-41ae-bb83-99f67bc8f0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:22.410058 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f858a4ac-fe83-41ae-bb83-99f67bc8f0e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:22.414958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f858a4ac-fe83-41ae-bb83-99f67bc8f0e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:22.422035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f858a4ac-fe83-41ae-bb83-99f67bc8f0e3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:22.498040 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 901/1793] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:05:22 2022] POST /identity/v3/projects => generated 377 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:22.503087 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec589f6-5928-4cd4-a961-6fd9a50bdab4 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:22.522579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec589f6-5928-4cd4-a961-6fd9a50bdab4 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:22.600461 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2d6fe-5824-464e-a512-9d14bc0fcb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:22.601308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2d6fe-5824-464e-a512-9d14bc0fcb5d admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:22.601753 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2d6fe-5824-464e-a512-9d14bc0fcb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:22.602828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2d6fe-5824-464e-a512-9d14bc0fcb5d admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:22.602975 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc2d6fe-5824-464e-a512-9d14bc0fcb5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:22.606464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc2d6fe-5824-464e-a512-9d14bc0fcb5d admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:22.645670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 893/1794] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:05:22 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:22.651745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f858a4ac-fe83-41ae-bb83-99f67bc8f0e3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:22.665541 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f858a4ac-fe83-41ae-bb83-99f67bc8f0e3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:22.681640 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5e35d-82d8-4fd9-82a1-c921e1b9dcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:22.682759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5e35d-82d8-4fd9-82a1-c921e1b9dcbf admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:22.682759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5e35d-82d8-4fd9-82a1-c921e1b9dcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:22.683239 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5e35d-82d8-4fd9-82a1-c921e1b9dcbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:22.684148 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e5e35d-82d8-4fd9-82a1-c921e1b9dcbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:22.687213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e5e35d-82d8-4fd9-82a1-c921e1b9dcbf admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:22.712399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 902/1795] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:22.715882 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2d6fe-5824-464e-a512-9d14bc0fcb5d admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:22.727839 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2d6fe-5824-464e-a512-9d14bc0fcb5d admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:22.734589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b07c5-46fd-4c92-8188-785ed5657d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:22.735322 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b07c5-46fd-4c92-8188-785ed5657d4a admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:22.744238 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b07c5-46fd-4c92-8188-785ed5657d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:22.744924 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b07c5-46fd-4c92-8188-785ed5657d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:22.747641 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b07c5-46fd-4c92-8188-785ed5657d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:22.752772 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b07c5-46fd-4c92-8188-785ed5657d4a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:22.773175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 894/1796] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:22.777017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5e35d-82d8-4fd9-82a1-c921e1b9dcbf admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:22.789738 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5e35d-82d8-4fd9-82a1-c921e1b9dcbf admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:22.795197 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ca924-db49-4af0-be06-24781de4fcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:22.795932 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ca924-db49-4af0-be06-24781de4fcc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:22.796803 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ca924-db49-4af0-be06-24781de4fcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:22.796803 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ca924-db49-4af0-be06-24781de4fcc5 admin admin] PATH_INFO: `/v3/projects/32c9b9e47a43415aa9cc2b083c1ad8e8/users/e2a1a9731319485885db076102025e46/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:22.818682 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ca924-db49-4af0-be06-24781de4fcc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32c9b9e47a43415aa9cc2b083c1ad8e8, user_id=e2a1a9731319485885db076102025e46, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:22.822015 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ca924-db49-4af0-be06-24781de4fcc5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:22.839267 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 903/1797] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:22 2022] PUT /identity/v3/projects/32c9b9e47a43415aa9cc2b083c1ad8e8/users/e2a1a9731319485885db076102025e46/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:22.850121 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3c855-9e96-45dc-b831-9216c6e8dff9 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:22.851475 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3c855-9e96-45dc-b831-9216c6e8dff9 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:22.852312 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3c855-9e96-45dc-b831-9216c6e8dff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:22.902897 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-bae3c855-9e96-45dc-b831-9216c6e8dff9 None None] MFA Rules not processed for user `e2a1a9731319485885db076102025e46`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:22.904435 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bae3c855-9e96-45dc-b831-9216c6e8dff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:23.003696 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 895/1798] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:23.006873 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ca924-db49-4af0-be06-24781de4fcc5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:23.103463 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ca924-db49-4af0-be06-24781de4fcc5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:23.110790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680272af-48c0-4c57-b815-fd154e29fb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:23.111601 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680272af-48c0-4c57-b815-fd154e29fb00 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:23.112344 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680272af-48c0-4c57-b815-fd154e29fb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:23.112344 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680272af-48c0-4c57-b815-fd154e29fb00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:23.113253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680272af-48c0-4c57-b815-fd154e29fb00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:23.116856 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680272af-48c0-4c57-b815-fd154e29fb00 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:23.134273 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3c855-9e96-45dc-b831-9216c6e8dff9 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:23.138133 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bae3c855-9e96-45dc-b831-9216c6e8dff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:23.161486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3c855-9e96-45dc-b831-9216c6e8dff9 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:23.169476 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4192910-7aef-40dc-910f-a0874f96df8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:23.170607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4192910-7aef-40dc-910f-a0874f96df8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:23.170607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4192910-7aef-40dc-910f-a0874f96df8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:23.171593 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4192910-7aef-40dc-910f-a0874f96df8b admin admin] PATH_INFO: `/v3/projects/687dccccc3b54cb0a45833789544937a` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:23.173307 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4192910-7aef-40dc-910f-a0874f96df8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=687dccccc3b54cb0a45833789544937a)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:23.175756 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4192910-7aef-40dc-910f-a0874f96df8b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:23.215529 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:23.216468 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 904/1799] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:05:23 2022] POST /identity/v3/projects => generated 377 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:23.222485 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680272af-48c0-4c57-b815-fd154e29fb00 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:23.291946 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:23.308004 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b4192910-7aef-40dc-910f-a0874f96df8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '687dccccc3b54cb0a45833789544937a', 'request_id': 'req-b4192910-7aef-40dc-910f-a0874f96df8b'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:23.320514 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b4192910-7aef-40dc-910f-a0874f96df8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '687dccccc3b54cb0a45833789544937a', 'request_id': 'req-b4192910-7aef-40dc-910f-a0874f96df8b'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:23.327156 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b4192910-7aef-40dc-910f-a0874f96df8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '687dccccc3b54cb0a45833789544937a', 'request_id': 'req-b4192910-7aef-40dc-910f-a0874f96df8b'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:23.342209 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b4192910-7aef-40dc-910f-a0874f96df8b admin admin] The token cache is being invalidate because project 687dccccc3b54cb0a45833789544937a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:23.342505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b4192910-7aef-40dc-910f-a0874f96df8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:23.343209 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b4192910-7aef-40dc-910f-a0874f96df8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:23.344956 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 896/1800] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:23 2022] DELETE /identity/v3/projects/687dccccc3b54cb0a45833789544937a => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:23.350541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4016ff6-923b-490b-82cc-6b5c3f47f643 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:23.350997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4016ff6-923b-490b-82cc-6b5c3f47f643 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:23.351288 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4016ff6-923b-490b-82cc-6b5c3f47f643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:23.359920 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680272af-48c0-4c57-b815-fd154e29fb00 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:23.376823 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a4016ff6-923b-490b-82cc-6b5c3f47f643 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:23.378129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a4016ff6-923b-490b-82cc-6b5c3f47f643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:23.439797 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdaccf1-ec11-4b04-99d2-029f8109a1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:23.440410 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdaccf1-ec11-4b04-99d2-029f8109a1a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:23.440787 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdaccf1-ec11-4b04-99d2-029f8109a1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:23.441149 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdaccf1-ec11-4b04-99d2-029f8109a1a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:23.442131 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdaccf1-ec11-4b04-99d2-029f8109a1a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:23.449966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdaccf1-ec11-4b04-99d2-029f8109a1a9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:23.482299 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 897/1801] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:05:23.487603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 905/1802] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:05:23 2022] POST /identity/v3/users => generated 295 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:23.491264 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdaccf1-ec11-4b04-99d2-029f8109a1a9 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:23.491585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ae896-208f-49c3-89ac-7eaf48f511c7 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:23.491885 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ae896-208f-49c3-89ac-7eaf48f511c7 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:23.492171 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ae896-208f-49c3-89ac-7eaf48f511c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:23.493066 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ccdaccf1-ec11-4b04-99d2-029f8109a1a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:23.504782 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdaccf1-ec11-4b04-99d2-029f8109a1a9 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:23.510334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8a0ae-5410-4da3-8008-d1cc169a8eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:23.510886 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8a0ae-5410-4da3-8008-d1cc169a8eaa admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:23.511168 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8a0ae-5410-4da3-8008-d1cc169a8eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:23.511443 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8a0ae-5410-4da3-8008-d1cc169a8eaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:23.511847 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-c25ae896-208f-49c3-89ac-7eaf48f511c7 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:23.512115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8a0ae-5410-4da3-8008-d1cc169a8eaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:23.513067 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c25ae896-208f-49c3-89ac-7eaf48f511c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:23.513711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8a0ae-5410-4da3-8008-d1cc169a8eaa admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:23.525138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 898/1803] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:23.525605 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 906/1804] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:23.529285 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8a0ae-5410-4da3-8008-d1cc169a8eaa admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:23.529422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ae896-208f-49c3-89ac-7eaf48f511c7 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:23.540826 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8a0ae-5410-4da3-8008-d1cc169a8eaa admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:23.541347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ae896-208f-49c3-89ac-7eaf48f511c7 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:23.546691 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d15b8-d9f2-4d58-b26e-dbdfdf3f19ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:23.547313 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d15b8-d9f2-4d58-b26e-dbdfdf3f19ab admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:23.547710 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d15b8-d9f2-4d58-b26e-dbdfdf3f19ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:23.547997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d15b8-d9f2-4d58-b26e-dbdfdf3f19ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:23.548420 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d64bce-aa0a-429d-a255-3cf8a6965665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:23.548644 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d15b8-d9f2-4d58-b26e-dbdfdf3f19ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:05:23.548948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d15b8-d9f2-4d58-b26e-dbdfdf3f19ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:23.549329 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d64bce-aa0a-429d-a255-3cf8a6965665 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:23.549677 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d64bce-aa0a-429d-a255-3cf8a6965665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:23.550160 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d64bce-aa0a-429d-a255-3cf8a6965665 admin admin] PATH_INFO: `/v3/projects/d8ee4628e9f349369ea76d4e9d399733/users/b33eff288a194ccbaf8aa7aeb9ceddd1/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:23.550702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d15b8-d9f2-4d58-b26e-dbdfdf3f19ab admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:23.568764 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 899/1805] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:05:23.569816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d15b8-d9f2-4d58-b26e-dbdfdf3f19ab admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:23.571758 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d64bce-aa0a-429d-a255-3cf8a6965665 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8ee4628e9f349369ea76d4e9d399733, user_id=b33eff288a194ccbaf8aa7aeb9ceddd1, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:23.574733 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d64bce-aa0a-429d-a255-3cf8a6965665 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:23.581163 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d15b8-d9f2-4d58-b26e-dbdfdf3f19ab admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:23.586903 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00ab67-0fd2-403c-af47-c38f0bda1297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:23.587510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00ab67-0fd2-403c-af47-c38f0bda1297 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:23.587821 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00ab67-0fd2-403c-af47-c38f0bda1297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:23.588096 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00ab67-0fd2-403c-af47-c38f0bda1297 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:23.588730 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00ab67-0fd2-403c-af47-c38f0bda1297 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:23.590743 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f00ab67-0fd2-403c-af47-c38f0bda1297 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:23.595381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 907/1806] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:23 2022] PUT /identity/v3/projects/d8ee4628e9f349369ea76d4e9d399733/users/b33eff288a194ccbaf8aa7aeb9ceddd1/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:23.598914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d64bce-aa0a-429d-a255-3cf8a6965665 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:23.601380 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-93d64bce-aa0a-429d-a255-3cf8a6965665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:23.620125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 900/1807] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:05:23 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:05:23.620952 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00ab67-0fd2-403c-af47-c38f0bda1297 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:23.671012 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d64bce-aa0a-429d-a255-3cf8a6965665 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:23.684526 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00ab67-0fd2-403c-af47-c38f0bda1297 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:23.690093 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32a5fb-194f-4d80-b33c-fafa4bf7a9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:23.690976 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32a5fb-194f-4d80-b33c-fafa4bf7a9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:23.691267 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32a5fb-194f-4d80-b33c-fafa4bf7a9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:23.691580 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32a5fb-194f-4d80-b33c-fafa4bf7a9a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:23.692226 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32a5fb-194f-4d80-b33c-fafa4bf7a9a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:23.694591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32a5fb-194f-4d80-b33c-fafa4bf7a9a6 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:23.697460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:23.705115 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 901/1808] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:23.705908 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32a5fb-194f-4d80-b33c-fafa4bf7a9a6 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:23.710748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:23.717384 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32a5fb-194f-4d80-b33c-fafa4bf7a9a6 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:23.722988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23aa01-c2ca-4303-829d-c651af1eef49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:23.723730 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23aa01-c2ca-4303-829d-c651af1eef49 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:23.724123 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23aa01-c2ca-4303-829d-c651af1eef49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:23.725165 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23aa01-c2ca-4303-829d-c651af1eef49 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:23.726020 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23aa01-c2ca-4303-829d-c651af1eef49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:23.728025 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23aa01-c2ca-4303-829d-c651af1eef49 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:23.733069 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bfa732-6a86-4aa5-9b61-8207888531e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:23.733711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfa732-6a86-4aa5-9b61-8207888531e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:23.734023 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfa732-6a86-4aa5-9b61-8207888531e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:23.734301 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfa732-6a86-4aa5-9b61-8207888531e7 admin admin] PATH_INFO: `/v3/users/18aa833ccad3454aae1fd89f6e26bbae` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:23.735500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bfa732-6a86-4aa5-9b61-8207888531e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18aa833ccad3454aae1fd89f6e26bbae)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:23.737450 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bfa732-6a86-4aa5-9b61-8207888531e7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:23.749226 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 902/1809] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:05:23 2022] POST /identity/v3/users => generated 285 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:23.750027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23aa01-c2ca-4303-829d-c651af1eef49 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:23.760523 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23aa01-c2ca-4303-829d-c651af1eef49 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:23.765728 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35ea2f-2101-4361-a83a-335e7ec1ff8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:23.766447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35ea2f-2101-4361-a83a-335e7ec1ff8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:23.766728 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35ea2f-2101-4361-a83a-335e7ec1ff8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:23.766998 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35ea2f-2101-4361-a83a-335e7ec1ff8f admin admin] PATH_INFO: `/v3/projects/d8ee4628e9f349369ea76d4e9d399733/users/b33eff288a194ccbaf8aa7aeb9ceddd1/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:23.769781 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e35ea2f-2101-4361-a83a-335e7ec1ff8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8ee4628e9f349369ea76d4e9d399733, user_id=b33eff288a194ccbaf8aa7aeb9ceddd1, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:23.773249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e35ea2f-2101-4361-a83a-335e7ec1ff8f admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:23.785916 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 903/1810] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:23 2022] PUT /identity/v3/projects/d8ee4628e9f349369ea76d4e9d399733/users/b33eff288a194ccbaf8aa7aeb9ceddd1/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:23.787323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35ea2f-2101-4361-a83a-335e7ec1ff8f admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:23.792578 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-16bfa732-6a86-4aa5-9b61-8207888531e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18aa833ccad3454aae1fd89f6e26bbae'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:23.793436 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-16bfa732-6a86-4aa5-9b61-8207888531e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18aa833ccad3454aae1fd89f6e26bbae'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:23.801636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-16bfa732-6a86-4aa5-9b61-8207888531e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18aa833ccad3454aae1fd89f6e26bbae'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:23.802957 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-16bfa732-6a86-4aa5-9b61-8207888531e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18aa833ccad3454aae1fd89f6e26bbae'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:23.810600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-16bfa732-6a86-4aa5-9b61-8207888531e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18aa833ccad3454aae1fd89f6e26bbae'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:23.828504 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 908/1811] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:23 2022] DELETE /identity/v3/users/18aa833ccad3454aae1fd89f6e26bbae => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:23.829732 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f13a8-218b-4519-a1cc-6d96451309a9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:23.830013 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f13a8-218b-4519-a1cc-6d96451309a9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:23.830289 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f13a8-218b-4519-a1cc-6d96451309a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:23.856291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35ea2f-2101-4361-a83a-335e7ec1ff8f admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:23.859761 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-aa8f13a8-218b-4519-a1cc-6d96451309a9 None None] MFA Rules not processed for user `b33eff288a194ccbaf8aa7aeb9ceddd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:23.860980 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-aa8f13a8-218b-4519-a1cc-6d96451309a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:23.923357 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58878ffb-1233-4364-93c1-7d212e6ac1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:23.924018 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58878ffb-1233-4364-93c1-7d212e6ac1b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:23.924298 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58878ffb-1233-4364-93c1-7d212e6ac1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:23.924688 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58878ffb-1233-4364-93c1-7d212e6ac1b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:23.925334 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58878ffb-1233-4364-93c1-7d212e6ac1b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:23.927844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58878ffb-1233-4364-93c1-7d212e6ac1b1 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:23.938581 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 909/1812] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:23 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:23.942197 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 904/1813] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:23.942331 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f7401-d795-47ee-a7fe-f3fbab4232a3 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:23.942331 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f7401-d795-47ee-a7fe-f3fbab4232a3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:23.942331 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f7401-d795-47ee-a7fe-f3fbab4232a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:23.944048 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58878ffb-1233-4364-93c1-7d212e6ac1b1 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:23.945941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-58878ffb-1233-4364-93c1-7d212e6ac1b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:23.963402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58878ffb-1233-4364-93c1-7d212e6ac1b1 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:23.968917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e33888-7d2a-445d-a39e-68ba3186b617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:23.969079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-330f7401-d795-47ee-a7fe-f3fbab4232a3 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:23.969541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e33888-7d2a-445d-a39e-68ba3186b617 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:23.969816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e33888-7d2a-445d-a39e-68ba3186b617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:23.970148 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e33888-7d2a-445d-a39e-68ba3186b617 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:23.970257 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-330f7401-d795-47ee-a7fe-f3fbab4232a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:23.970797 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e33888-7d2a-445d-a39e-68ba3186b617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:23.972535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e33888-7d2a-445d-a39e-68ba3186b617 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:23.981682 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 910/1814] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:23.981946 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f7401-d795-47ee-a7fe-f3fbab4232a3 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:23.983535 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-330f7401-d795-47ee-a7fe-f3fbab4232a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:23.997169 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 905/1815] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:05:23.997169 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e33888-7d2a-445d-a39e-68ba3186b617 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:23.998330 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-96e33888-7d2a-445d-a39e-68ba3186b617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:24.041556 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:24.074489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e33888-7d2a-445d-a39e-68ba3186b617 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:24.078431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5d8d7-310d-4d93-8aa9-63b83396205c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:24.078602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f7401-d795-47ee-a7fe-f3fbab4232a3 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:24.079055 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5d8d7-310d-4d93-8aa9-63b83396205c service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:24.079335 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5d8d7-310d-4d93-8aa9-63b83396205c service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:24.079649 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5d8d7-310d-4d93-8aa9-63b83396205c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:24.083385 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc3dd70-916a-4921-90e8-15ff98e125b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:24.085106 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc3dd70-916a-4921-90e8-15ff98e125b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:24.085697 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc3dd70-916a-4921-90e8-15ff98e125b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:24.086290 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc3dd70-916a-4921-90e8-15ff98e125b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:24.090831 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a5d8d7-310d-4d93-8aa9-63b83396205c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:24.092604 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a5d8d7-310d-4d93-8aa9-63b83396205c service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:24.098225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc3dd70-916a-4921-90e8-15ff98e125b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:24.100542 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc3dd70-916a-4921-90e8-15ff98e125b5 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:24.113877 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 911/1816] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:23 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:24.115416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc3dd70-916a-4921-90e8-15ff98e125b5 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:24.121279 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 906/1817] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:24.137671 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc3dd70-916a-4921-90e8-15ff98e125b5 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:24.144854 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2390304-ec96-4860-a263-a0dbcf72e002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:24.145537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2390304-ec96-4860-a263-a0dbcf72e002 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:24.145906 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2390304-ec96-4860-a263-a0dbcf72e002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:24.146214 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2390304-ec96-4860-a263-a0dbcf72e002 admin admin] PATH_INFO: `/v3/projects/5b8d4d4698b0402f94f563412ddab915/users/4bc38a3e315c41369948889260533120/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:24.176952 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5d8d7-310d-4d93-8aa9-63b83396205c service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:24.179410 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-21a5d8d7-310d-4d93-8aa9-63b83396205c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:24.195284 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2390304-ec96-4860-a263-a0dbcf72e002 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b8d4d4698b0402f94f563412ddab915, user_id=4bc38a3e315c41369948889260533120, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:24.196293 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5d8d7-310d-4d93-8aa9-63b83396205c service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:24.200775 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2390304-ec96-4860-a263-a0dbcf72e002 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:24.201734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf91b66-6e19-4a86-9162-60c4061c62cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:24.202465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf91b66-6e19-4a86-9162-60c4061c62cf service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:24.202889 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf91b66-6e19-4a86-9162-60c4061c62cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:24.203267 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf91b66-6e19-4a86-9162-60c4061c62cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:24.205539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ccf91b66-6e19-4a86-9162-60c4061c62cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:24.212331 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:24.224758 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 912/1818] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:24 2022] PUT /identity/v3/projects/5b8d4d4698b0402f94f563412ddab915/users/4bc38a3e315c41369948889260533120/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:24.236150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d27f1e-4996-4e21-b65e-2fefb21c1cb9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:24.236698 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d27f1e-4996-4e21-b65e-2fefb21c1cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:24.237144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d27f1e-4996-4e21-b65e-2fefb21c1cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:24.273260 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-20d27f1e-4996-4e21-b65e-2fefb21c1cb9 None None] MFA Rules not processed for user `4bc38a3e315c41369948889260533120`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:24.274815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-20d27f1e-4996-4e21-b65e-2fefb21c1cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:24.288545 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf91b66-6e19-4a86-9162-60c4061c62cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:24.290177 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf91b66-6e19-4a86-9162-60c4061c62cf service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:24.310965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 907/1819] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:05:24.387592 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 913/1820] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:24.388803 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf91b66-6e19-4a86-9162-60c4061c62cf service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:24.405724 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf91b66-6e19-4a86-9162-60c4061c62cf service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:24.500087 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d4ed3-bce2-402d-9333-23199fc0a697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:24.500733 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d4ed3-bce2-402d-9333-23199fc0a697 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:24.501019 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d4ed3-bce2-402d-9333-23199fc0a697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:24.501298 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d4ed3-bce2-402d-9333-23199fc0a697 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:24.502039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d4ed3-bce2-402d-9333-23199fc0a697 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:24.504416 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d4ed3-bce2-402d-9333-23199fc0a697 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:24.544935 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 908/1821] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:05:24 2022] POST /identity/v3/projects => generated 369 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:24.548826 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d27f1e-4996-4e21-b65e-2fefb21c1cb9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:24.564658 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d27f1e-4996-4e21-b65e-2fefb21c1cb9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:24.662914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a590b37-43ad-49a3-a47a-cf6e52260937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:24.664011 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a590b37-43ad-49a3-a47a-cf6e52260937 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:24.664364 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a590b37-43ad-49a3-a47a-cf6e52260937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:24.664725 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a590b37-43ad-49a3-a47a-cf6e52260937 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:24.665576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a590b37-43ad-49a3-a47a-cf6e52260937 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:24.667942 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a590b37-43ad-49a3-a47a-cf6e52260937 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:24.706905 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 914/1822] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:05:24 2022] POST /identity/v3/users => generated 291 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:24.708230 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d4ed3-bce2-402d-9333-23199fc0a697 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:24.730107 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d4ed3-bce2-402d-9333-23199fc0a697 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:24.735497 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b6b9a-62cc-450a-bd6a-75cd8a5eee6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:24.735774 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b6b9a-62cc-450a-bd6a-75cd8a5eee6c admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:24.736125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b6b9a-62cc-450a-bd6a-75cd8a5eee6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:24.736475 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b6b9a-62cc-450a-bd6a-75cd8a5eee6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:24.737361 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b6b9a-62cc-450a-bd6a-75cd8a5eee6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:24.740901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b6b9a-62cc-450a-bd6a-75cd8a5eee6c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:24.769615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 909/1823] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:24.774823 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a590b37-43ad-49a3-a47a-cf6e52260937 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:24.785629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a590b37-43ad-49a3-a47a-cf6e52260937 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:24.804735 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff2086-a42c-4277-9d97-4b714a55797a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:24.805647 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff2086-a42c-4277-9d97-4b714a55797a admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:24.806105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff2086-a42c-4277-9d97-4b714a55797a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:24.807096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff2086-a42c-4277-9d97-4b714a55797a admin admin] PATH_INFO: `/v3/projects/b08679cb668c4533b886b66f4fd20fde/users/2ff523319eb74ebf8b298d9a8b3e79cd/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:24.837009 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff2086-a42c-4277-9d97-4b714a55797a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b08679cb668c4533b886b66f4fd20fde, user_id=2ff523319eb74ebf8b298d9a8b3e79cd, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:24.839527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff2086-a42c-4277-9d97-4b714a55797a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:24.874650 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:24.880218 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 915/1824] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:24 2022] PUT /identity/v3/projects/b08679cb668c4533b886b66f4fd20fde/users/2ff523319eb74ebf8b298d9a8b3e79cd/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:24.885118 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b6b9a-62cc-450a-bd6a-75cd8a5eee6c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:24.898376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b6b9a-62cc-450a-bd6a-75cd8a5eee6c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:24.983592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682dc49-a0a5-428b-97f8-e9bee7d2d4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:24.984474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682dc49-a0a5-428b-97f8-e9bee7d2d4dd admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:24.985345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682dc49-a0a5-428b-97f8-e9bee7d2d4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:24.985631 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682dc49-a0a5-428b-97f8-e9bee7d2d4dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:24.986285 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682dc49-a0a5-428b-97f8-e9bee7d2d4dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:24.988151 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682dc49-a0a5-428b-97f8-e9bee7d2d4dd admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:25.013476 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 910/1825] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:25.021520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff2086-a42c-4277-9d97-4b714a55797a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:25.033203 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff2086-a42c-4277-9d97-4b714a55797a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:25.042994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e498e5d6-a558-4efc-b24c-d5dea533ad01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:25.044093 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498e5d6-a558-4efc-b24c-d5dea533ad01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:25.044458 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498e5d6-a558-4efc-b24c-d5dea533ad01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:25.044825 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498e5d6-a558-4efc-b24c-d5dea533ad01 admin admin] PATH_INFO: `/v3/projects/b08679cb668c4533b886b66f4fd20fde/users/2ff523319eb74ebf8b298d9a8b3e79cd/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:25.047008 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e498e5d6-a558-4efc-b24c-d5dea533ad01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b08679cb668c4533b886b66f4fd20fde, user_id=2ff523319eb74ebf8b298d9a8b3e79cd, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:25.050289 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e498e5d6-a558-4efc-b24c-d5dea533ad01 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:25.078411 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682dc49-a0a5-428b-97f8-e9bee7d2d4dd admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:25.080313 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d682dc49-a0a5-428b-97f8-e9bee7d2d4dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:25.084234 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 916/1826] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:25 2022] PUT /identity/v3/projects/b08679cb668c4533b886b66f4fd20fde/users/2ff523319eb74ebf8b298d9a8b3e79cd/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:25.085872 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc3025-7c3b-41cc-a3e4-411bb05e1f1c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:25.086237 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc3025-7c3b-41cc-a3e4-411bb05e1f1c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:25.086590 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc3025-7c3b-41cc-a3e4-411bb05e1f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:25.115369 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682dc49-a0a5-428b-97f8-e9bee7d2d4dd admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:25.126146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-6efc3025-7c3b-41cc-a3e4-411bb05e1f1c None None] MFA Rules not processed for user `2ff523319eb74ebf8b298d9a8b3e79cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:25.127807 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6efc3025-7c3b-41cc-a3e4-411bb05e1f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:25.147731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:25.239083 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f1c1a-21f7-497f-9c69-cd24b6d78cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:25.240050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f1c1a-21f7-497f-9c69-cd24b6d78cd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:25.240359 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f1c1a-21f7-497f-9c69-cd24b6d78cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:25.240642 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f1c1a-21f7-497f-9c69-cd24b6d78cd5 admin admin] PATH_INFO: `/v3/projects/11294376d60849ae8256930c6630aa24` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:25.241689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 917/1827] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:25 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:25.241972 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f1c1a-21f7-497f-9c69-cd24b6d78cd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11294376d60849ae8256930c6630aa24)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:25.244309 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f1c1a-21f7-497f-9c69-cd24b6d78cd5 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:25.259308 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc3025-7c3b-41cc-a3e4-411bb05e1f1c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:25.329900 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc3025-7c3b-41cc-a3e4-411bb05e1f1c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:25.342362 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edecba5-fbe1-4070-aa92-8207f015fc57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:25.343005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edecba5-fbe1-4070-aa92-8207f015fc57 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:25.343285 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edecba5-fbe1-4070-aa92-8207f015fc57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:25.343579 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edecba5-fbe1-4070-aa92-8207f015fc57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:25.373944 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edecba5-fbe1-4070-aa92-8207f015fc57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:25.375408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edecba5-fbe1-4070-aa92-8207f015fc57 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:25.391159 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-5a6f1c1a-21f7-497f-9c69-cd24b6d78cd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11294376d60849ae8256930c6630aa24', 'request_id': 'req-5a6f1c1a-21f7-497f-9c69-cd24b6d78cd5'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:25.400135 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 918/1828] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:25 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:25.415370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-5a6f1c1a-21f7-497f-9c69-cd24b6d78cd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11294376d60849ae8256930c6630aa24', 'request_id': 'req-5a6f1c1a-21f7-497f-9c69-cd24b6d78cd5'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:25.424536 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-5a6f1c1a-21f7-497f-9c69-cd24b6d78cd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11294376d60849ae8256930c6630aa24', 'request_id': 'req-5a6f1c1a-21f7-497f-9c69-cd24b6d78cd5'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:25.438062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-5a6f1c1a-21f7-497f-9c69-cd24b6d78cd5 admin admin] The token cache is being invalidate because project 11294376d60849ae8256930c6630aa24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:25.438433 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-5a6f1c1a-21f7-497f-9c69-cd24b6d78cd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:25.439167 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-5a6f1c1a-21f7-497f-9c69-cd24b6d78cd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:25.441177 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 911/1829] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:25 2022] DELETE /identity/v3/projects/11294376d60849ae8256930c6630aa24 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:05:25.445618 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edecba5-fbe1-4070-aa92-8207f015fc57 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:25.456233 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edecba5-fbe1-4070-aa92-8207f015fc57 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:25.524838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc452b5d-7d48-462d-b3b9-9df527afee00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:25.525208 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc452b5d-7d48-462d-b3b9-9df527afee00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:25.525580 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc452b5d-7d48-462d-b3b9-9df527afee00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:25.525920 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc452b5d-7d48-462d-b3b9-9df527afee00 admin admin] PATH_INFO: `/v3/users/1564db0754a54c7caba26274574d823e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:25.527151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc452b5d-7d48-462d-b3b9-9df527afee00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1564db0754a54c7caba26274574d823e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:25.529265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc452b5d-7d48-462d-b3b9-9df527afee00 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:25.554895 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:25.600585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bc452b5d-7d48-462d-b3b9-9df527afee00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1564db0754a54c7caba26274574d823e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:25.601520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bc452b5d-7d48-462d-b3b9-9df527afee00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1564db0754a54c7caba26274574d823e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:25.610768 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bc452b5d-7d48-462d-b3b9-9df527afee00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1564db0754a54c7caba26274574d823e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:25.611762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bc452b5d-7d48-462d-b3b9-9df527afee00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1564db0754a54c7caba26274574d823e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:25.625406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bc452b5d-7d48-462d-b3b9-9df527afee00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1564db0754a54c7caba26274574d823e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:25.656409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 919/1830] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:25 2022] DELETE /identity/v3/users/1564db0754a54c7caba26274574d823e => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:26.432076 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f1c1a-21f7-497f-9c69-cd24b6d78cd5 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:26.432076 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5a6f1c1a-21f7-497f-9c69-cd24b6d78cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:26.452194 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f1c1a-21f7-497f-9c69-cd24b6d78cd5 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:26.619172 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9602ae-ff9e-4696-a859-3865cf503c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:26.620916 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9602ae-ff9e-4696-a859-3865cf503c0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:26.623392 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9602ae-ff9e-4696-a859-3865cf503c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:26.624672 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9602ae-ff9e-4696-a859-3865cf503c0a admin admin] PATH_INFO: `/v3/projects/73aa6c4496a74ce3a59ee8846f8f4364` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:26.627850 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9602ae-ff9e-4696-a859-3865cf503c0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=73aa6c4496a74ce3a59ee8846f8f4364)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:26.630069 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9602ae-ff9e-4696-a859-3865cf503c0a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:26.735690 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:26.737859 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9d9602ae-ff9e-4696-a859-3865cf503c0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73aa6c4496a74ce3a59ee8846f8f4364', 'request_id': 'req-9d9602ae-ff9e-4696-a859-3865cf503c0a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:26.751878 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9d9602ae-ff9e-4696-a859-3865cf503c0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73aa6c4496a74ce3a59ee8846f8f4364', 'request_id': 'req-9d9602ae-ff9e-4696-a859-3865cf503c0a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:26.778557 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9d9602ae-ff9e-4696-a859-3865cf503c0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73aa6c4496a74ce3a59ee8846f8f4364', 'request_id': 'req-9d9602ae-ff9e-4696-a859-3865cf503c0a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:26.800270 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9d9602ae-ff9e-4696-a859-3865cf503c0a admin admin] The token cache is being invalidate because project 73aa6c4496a74ce3a59ee8846f8f4364 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:26.800733 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9d9602ae-ff9e-4696-a859-3865cf503c0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:26.801589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-9d9602ae-ff9e-4696-a859-3865cf503c0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:26.803999 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 912/1831] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:26 2022] DELETE /identity/v3/projects/73aa6c4496a74ce3a59ee8846f8f4364 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:26.808582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc452b5d-7d48-462d-b3b9-9df527afee00 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:26.810489 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bc452b5d-7d48-462d-b3b9-9df527afee00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:26.826589 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc452b5d-7d48-462d-b3b9-9df527afee00 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:26.930893 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896722b-2e60-4d60-87ea-427874eefa48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:26.932337 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896722b-2e60-4d60-87ea-427874eefa48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:26.932739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896722b-2e60-4d60-87ea-427874eefa48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:26.933546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896722b-2e60-4d60-87ea-427874eefa48 admin admin] PATH_INFO: `/v3/users/7828dfc837724c3dada86fa1303ea48c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:26.935011 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896722b-2e60-4d60-87ea-427874eefa48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7828dfc837724c3dada86fa1303ea48c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:26.937919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896722b-2e60-4d60-87ea-427874eefa48 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:27.002204 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0896722b-2e60-4d60-87ea-427874eefa48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7828dfc837724c3dada86fa1303ea48c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:27.002874 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0896722b-2e60-4d60-87ea-427874eefa48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7828dfc837724c3dada86fa1303ea48c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:27.011858 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0896722b-2e60-4d60-87ea-427874eefa48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7828dfc837724c3dada86fa1303ea48c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:27.012471 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0896722b-2e60-4d60-87ea-427874eefa48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7828dfc837724c3dada86fa1303ea48c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:27.020110 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0896722b-2e60-4d60-87ea-427874eefa48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7828dfc837724c3dada86fa1303ea48c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:27.042059 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 920/1832] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:26 2022] DELETE /identity/v3/users/7828dfc837724c3dada86fa1303ea48c => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:27.287042 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9602ae-ff9e-4696-a859-3865cf503c0a admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:27.288528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9d9602ae-ff9e-4696-a859-3865cf503c0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:27.301595 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9602ae-ff9e-4696-a859-3865cf503c0a admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:27.362668 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61a860-2dd1-44cc-8296-dc2963ca2b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:27.363034 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61a860-2dd1-44cc-8296-dc2963ca2b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:27.363312 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61a860-2dd1-44cc-8296-dc2963ca2b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:27.363610 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61a860-2dd1-44cc-8296-dc2963ca2b95 admin admin] PATH_INFO: `/v3/users/4bc38a3e315c41369948889260533120` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:27.365256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61a860-2dd1-44cc-8296-dc2963ca2b95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bc38a3e315c41369948889260533120)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:27.367258 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61a860-2dd1-44cc-8296-dc2963ca2b95 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:27.421266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-dc61a860-2dd1-44cc-8296-dc2963ca2b95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bc38a3e315c41369948889260533120'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:27.421977 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-dc61a860-2dd1-44cc-8296-dc2963ca2b95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bc38a3e315c41369948889260533120'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:27.430229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-dc61a860-2dd1-44cc-8296-dc2963ca2b95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bc38a3e315c41369948889260533120'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:27.437392 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-dc61a860-2dd1-44cc-8296-dc2963ca2b95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bc38a3e315c41369948889260533120'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:27.438045 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-dc61a860-2dd1-44cc-8296-dc2963ca2b95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bc38a3e315c41369948889260533120'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:27.456949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 913/1833] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:27 2022] DELETE /identity/v3/users/4bc38a3e315c41369948889260533120 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:27.461218 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecd49e-29ba-4ed3-a365-d03a34693b76 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:27.461576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecd49e-29ba-4ed3-a365-d03a34693b76 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:27.461917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecd49e-29ba-4ed3-a365-d03a34693b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:27.483757 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-75ecd49e-29ba-4ed3-a365-d03a34693b76 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:27.485034 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-75ecd49e-29ba-4ed3-a365-d03a34693b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:27.494857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:27.549094 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61a860-2dd1-44cc-8296-dc2963ca2b95 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:27.550527 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-dc61a860-2dd1-44cc-8296-dc2963ca2b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:27.557879 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 921/1834] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:27.567108 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61a860-2dd1-44cc-8296-dc2963ca2b95 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:27.569441 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecd49e-29ba-4ed3-a365-d03a34693b76 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:27.571049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-75ecd49e-29ba-4ed3-a365-d03a34693b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:27.581829 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd20503-99a3-41eb-aa59-e70ddc668ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:27.582489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd20503-99a3-41eb-aa59-e70ddc668ef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:27.582895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd20503-99a3-41eb-aa59-e70ddc668ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:27.583254 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd20503-99a3-41eb-aa59-e70ddc668ef5 admin admin] PATH_INFO: `/v3/projects/733ece650f4f4b50bb2bc6f155d0386c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:27.585326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd20503-99a3-41eb-aa59-e70ddc668ef5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=733ece650f4f4b50bb2bc6f155d0386c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:27.587779 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd20503-99a3-41eb-aa59-e70ddc668ef5 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:27.654136 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-cfd20503-99a3-41eb-aa59-e70ddc668ef5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '733ece650f4f4b50bb2bc6f155d0386c', 'request_id': 'req-cfd20503-99a3-41eb-aa59-e70ddc668ef5'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:27.656225 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=81,keystone:INSERT=2,keystone:DELETE=17 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:27.657094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecd49e-29ba-4ed3-a365-d03a34693b76 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:27.664956 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-cfd20503-99a3-41eb-aa59-e70ddc668ef5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '733ece650f4f4b50bb2bc6f155d0386c', 'request_id': 'req-cfd20503-99a3-41eb-aa59-e70ddc668ef5'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:27.671935 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-cfd20503-99a3-41eb-aa59-e70ddc668ef5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '733ece650f4f4b50bb2bc6f155d0386c', 'request_id': 'req-cfd20503-99a3-41eb-aa59-e70ddc668ef5'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:27.684751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-cfd20503-99a3-41eb-aa59-e70ddc668ef5 admin admin] The token cache is being invalidate because project 733ece650f4f4b50bb2bc6f155d0386c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:27.685041 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-cfd20503-99a3-41eb-aa59-e70ddc668ef5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:27.685716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-cfd20503-99a3-41eb-aa59-e70ddc668ef5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:27.687208 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 914/1835] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:27 2022] DELETE /identity/v3/projects/733ece650f4f4b50bb2bc6f155d0386c => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:05:27.691897 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61810535-0099-492e-bc64-9531fd3aedfe None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:27.692318 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61810535-0099-492e-bc64-9531fd3aedfe None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:27.693390 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61810535-0099-492e-bc64-9531fd3aedfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:27.716622 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-61810535-0099-492e-bc64-9531fd3aedfe None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:27.717826 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-61810535-0099-492e-bc64-9531fd3aedfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:27.737112 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c04ddd-1b98-421a-8855-d72ef76281d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:27.738753 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c04ddd-1b98-421a-8855-d72ef76281d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:27.739042 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c04ddd-1b98-421a-8855-d72ef76281d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:27.739323 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c04ddd-1b98-421a-8855-d72ef76281d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:27.743098 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-97c04ddd-1b98-421a-8855-d72ef76281d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:27.803145 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 915/1836] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:27.804282 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61810535-0099-492e-bc64-9531fd3aedfe None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:27.805733 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-61810535-0099-492e-bc64-9531fd3aedfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:27.818816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61810535-0099-492e-bc64-9531fd3aedfe None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:27.824152 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e3da8-3a4e-4383-820b-b76fa7c04eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:27.825578 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e3da8-3a4e-4383-820b-b76fa7c04eae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:27.826166 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e3da8-3a4e-4383-820b-b76fa7c04eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:27.826920 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e3da8-3a4e-4383-820b-b76fa7c04eae admin admin] PATH_INFO: `/v3/users/1f7b68b9544a416c832526f69d591a8d` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:27.828675 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e3da8-3a4e-4383-820b-b76fa7c04eae admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f7b68b9544a416c832526f69d591a8d)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:27.830759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e3da8-3a4e-4383-820b-b76fa7c04eae admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:27.872008 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c04ddd-1b98-421a-8855-d72ef76281d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:27.873826 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c04ddd-1b98-421a-8855-d72ef76281d8 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:27.890336 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 922/1837] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:27.892495 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f103c-bb10-4e67-8446-17c767e53f71 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:27.893079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f103c-bb10-4e67-8446-17c767e53f71 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:27.893572 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f103c-bb10-4e67-8446-17c767e53f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:27.916919 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b13e3da8-3a4e-4383-820b-b76fa7c04eae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f7b68b9544a416c832526f69d591a8d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:27.917781 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b13e3da8-3a4e-4383-820b-b76fa7c04eae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f7b68b9544a416c832526f69d591a8d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:27.918133 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-432f103c-bb10-4e67-8446-17c767e53f71 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:27.919917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-432f103c-bb10-4e67-8446-17c767e53f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:27.931008 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 923/1838] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:27.935382 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f103c-bb10-4e67-8446-17c767e53f71 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:27.940516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b13e3da8-3a4e-4383-820b-b76fa7c04eae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f7b68b9544a416c832526f69d591a8d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:27.948151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f103c-bb10-4e67-8446-17c767e53f71 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:27.951546 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b13e3da8-3a4e-4383-820b-b76fa7c04eae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f7b68b9544a416c832526f69d591a8d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:27.952726 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b13e3da8-3a4e-4383-820b-b76fa7c04eae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f7b68b9544a416c832526f69d591a8d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:27.964110 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885bfdc-4ac1-4626-ba08-6f64fb65242a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:27.964806 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885bfdc-4ac1-4626-ba08-6f64fb65242a admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:27.965092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885bfdc-4ac1-4626-ba08-6f64fb65242a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:27.965367 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885bfdc-4ac1-4626-ba08-6f64fb65242a admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:27.966405 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885bfdc-4ac1-4626-ba08-6f64fb65242a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:05:27.966714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885bfdc-4ac1-4626-ba08-6f64fb65242a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:27.968589 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885bfdc-4ac1-4626-ba08-6f64fb65242a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:27.976960 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 916/1839] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:27 2022] DELETE /identity/v3/users/1f7b68b9544a416c832526f69d591a8d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:27.982560 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df44ca-fb2b-4606-a969-a5b7f70aef13 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:27.983134 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df44ca-fb2b-4606-a969-a5b7f70aef13 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:27.983513 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df44ca-fb2b-4606-a969-a5b7f70aef13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:27.994412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:28.007834 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 924/1840] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:05:28.010454 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-82df44ca-fb2b-4606-a969-a5b7f70aef13 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:28.011794 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885bfdc-4ac1-4626-ba08-6f64fb65242a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:28.012363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-82df44ca-fb2b-4606-a969-a5b7f70aef13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:28.013503 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9885bfdc-4ac1-4626-ba08-6f64fb65242a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:28.042756 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885bfdc-4ac1-4626-ba08-6f64fb65242a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:28.120141 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519fd74a-7c8a-4789-8788-d4c1d70fb3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:28.121285 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519fd74a-7c8a-4789-8788-d4c1d70fb3ac admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:28.121573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519fd74a-7c8a-4789-8788-d4c1d70fb3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:28.121849 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519fd74a-7c8a-4789-8788-d4c1d70fb3ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:28.122494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519fd74a-7c8a-4789-8788-d4c1d70fb3ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:28.125478 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519fd74a-7c8a-4789-8788-d4c1d70fb3ac admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:28.126260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:28.167742 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 925/1841] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:05:28 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:05:28.168501 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 917/1842] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:28.168964 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519fd74a-7c8a-4789-8788-d4c1d70fb3ac admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:28.192612 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df44ca-fb2b-4606-a969-a5b7f70aef13 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:28.194758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-82df44ca-fb2b-4606-a969-a5b7f70aef13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:28.293504 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519fd74a-7c8a-4789-8788-d4c1d70fb3ac admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:28.300236 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cc973b-549a-4e46-a02c-434c91d7a9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:28.301060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc973b-549a-4e46-a02c-434c91d7a9a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:28.301441 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc973b-549a-4e46-a02c-434c91d7a9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:28.301814 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc973b-549a-4e46-a02c-434c91d7a9a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:28.302769 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cc973b-549a-4e46-a02c-434c91d7a9a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:28.305385 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cc973b-549a-4e46-a02c-434c91d7a9a2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:28.310293 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df44ca-fb2b-4606-a969-a5b7f70aef13 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:28.337959 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 926/1843] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:05:28 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:28.342017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cc973b-549a-4e46-a02c-434c91d7a9a2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:28.360827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cc973b-549a-4e46-a02c-434c91d7a9a2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:28.366474 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23a7f9-c6a1-4dc6-9973-1e55542a48aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:28.367242 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23a7f9-c6a1-4dc6-9973-1e55542a48aa admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:28.367522 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23a7f9-c6a1-4dc6-9973-1e55542a48aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:28.367848 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23a7f9-c6a1-4dc6-9973-1e55542a48aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:28.368634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f23a7f9-c6a1-4dc6-9973-1e55542a48aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:28.370235 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f23a7f9-c6a1-4dc6-9973-1e55542a48aa admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:28.385633 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 927/1844] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:28.386872 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23a7f9-c6a1-4dc6-9973-1e55542a48aa admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:28.395146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ed0cd-ca25-4c56-83da-fd7b8256c494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:28.396336 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ed0cd-ca25-4c56-83da-fd7b8256c494 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:28.396616 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ed0cd-ca25-4c56-83da-fd7b8256c494 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:28.396890 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ed0cd-ca25-4c56-83da-fd7b8256c494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:28.398191 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23a7f9-c6a1-4dc6-9973-1e55542a48aa admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:28.403219 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10e7a5-1bd7-4769-aa98-52f9f9397107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:28.404063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10e7a5-1bd7-4769-aa98-52f9f9397107 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:28.404412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10e7a5-1bd7-4769-aa98-52f9f9397107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:28.404778 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10e7a5-1bd7-4769-aa98-52f9f9397107 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:28.405531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc10e7a5-1bd7-4769-aa98-52f9f9397107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:28.407183 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc10e7a5-1bd7-4769-aa98-52f9f9397107 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:28.407377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ed0cd-ca25-4c56-83da-fd7b8256c494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:28.408845 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ed0cd-ca25-4c56-83da-fd7b8256c494 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:28.424221 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 918/1845] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:28.435018 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 928/1846] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:05:28.436443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ed0cd-ca25-4c56-83da-fd7b8256c494 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:28.450986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ed0cd-ca25-4c56-83da-fd7b8256c494 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:28.465696 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a82ad0-86c3-4ec4-868b-e026748d1410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:28.466373 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a82ad0-86c3-4ec4-868b-e026748d1410 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:28.466653 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a82ad0-86c3-4ec4-868b-e026748d1410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:28.466930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a82ad0-86c3-4ec4-868b-e026748d1410 admin admin] PATH_INFO: `/v3/projects/111d427ecff1479781c0128156960807/users/84a61f841d8b48ddaecebce8469645e4/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:28.485273 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a82ad0-86c3-4ec4-868b-e026748d1410 admin admin] RBAC: Authorizing `identity:create_grant(project_id=111d427ecff1479781c0128156960807, user_id=84a61f841d8b48ddaecebce8469645e4, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:28.485406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10e7a5-1bd7-4769-aa98-52f9f9397107 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:28.487717 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cc10e7a5-1bd7-4769-aa98-52f9f9397107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:28.488835 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a82ad0-86c3-4ec4-868b-e026748d1410 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:28.505168 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10e7a5-1bd7-4769-aa98-52f9f9397107 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:28.515876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 919/1847] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:28 2022] PUT /identity/v3/projects/111d427ecff1479781c0128156960807/users/84a61f841d8b48ddaecebce8469645e4/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:05:28.522149 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d412b9-80d8-4eb0-a57d-25c57a013757 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:28.522652 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d412b9-80d8-4eb0-a57d-25c57a013757 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:28.522967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d412b9-80d8-4eb0-a57d-25c57a013757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:28.560074 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-b9d412b9-80d8-4eb0-a57d-25c57a013757 None None] MFA Rules not processed for user `84a61f841d8b48ddaecebce8469645e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:28.561338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b9d412b9-80d8-4eb0-a57d-25c57a013757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:28.579838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:28.607971 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e1c63-3012-4b46-8b36-20b3a01ed3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:28.608738 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e1c63-3012-4b46-8b36-20b3a01ed3a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:28.609180 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e1c63-3012-4b46-8b36-20b3a01ed3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:28.609521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e1c63-3012-4b46-8b36-20b3a01ed3a6 admin admin] PATH_INFO: `/v3/projects/5b8d4d4698b0402f94f563412ddab915` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:28.610875 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e1c63-3012-4b46-8b36-20b3a01ed3a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b8d4d4698b0402f94f563412ddab915)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:28.612900 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e1c63-3012-4b46-8b36-20b3a01ed3a6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:28.636420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 920/1848] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:05:28.641047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d412b9-80d8-4eb0-a57d-25c57a013757 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:28.656460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d412b9-80d8-4eb0-a57d-25c57a013757 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:28.672882 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2c7e1c63-3012-4b46-8b36-20b3a01ed3a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b8d4d4698b0402f94f563412ddab915', 'request_id': 'req-2c7e1c63-3012-4b46-8b36-20b3a01ed3a6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:28.673671 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:28.683838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2c7e1c63-3012-4b46-8b36-20b3a01ed3a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b8d4d4698b0402f94f563412ddab915', 'request_id': 'req-2c7e1c63-3012-4b46-8b36-20b3a01ed3a6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:28.709016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2c7e1c63-3012-4b46-8b36-20b3a01ed3a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b8d4d4698b0402f94f563412ddab915', 'request_id': 'req-2c7e1c63-3012-4b46-8b36-20b3a01ed3a6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:28.746052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2c7e1c63-3012-4b46-8b36-20b3a01ed3a6 admin admin] The token cache is being invalidate because project 5b8d4d4698b0402f94f563412ddab915 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:28.746399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2c7e1c63-3012-4b46-8b36-20b3a01ed3a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:28.747856 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2c7e1c63-3012-4b46-8b36-20b3a01ed3a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:28.751873 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3035fb-34ed-4bcd-914f-03665e4115e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:28.753061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 929/1849] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:28 2022] DELETE /identity/v3/projects/5b8d4d4698b0402f94f563412ddab915 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:28.753855 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3035fb-34ed-4bcd-914f-03665e4115e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:28.754366 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3035fb-34ed-4bcd-914f-03665e4115e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:28.754842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3035fb-34ed-4bcd-914f-03665e4115e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:28.756149 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3035fb-34ed-4bcd-914f-03665e4115e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:28.756829 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e1c63-3012-4b46-8b36-20b3a01ed3a6 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:28.759419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2c7e1c63-3012-4b46-8b36-20b3a01ed3a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:28.760462 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3035fb-34ed-4bcd-914f-03665e4115e4 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:28.780720 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e1c63-3012-4b46-8b36-20b3a01ed3a6 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:28.785939 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e37786-4e69-4cb4-b73b-b88b3b4f429c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:28.786925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e37786-4e69-4cb4-b73b-b88b3b4f429c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:28.787358 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e37786-4e69-4cb4-b73b-b88b3b4f429c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:28.787757 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e37786-4e69-4cb4-b73b-b88b3b4f429c admin admin] PATH_INFO: `/v3/users/2ff523319eb74ebf8b298d9a8b3e79cd` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:28.789090 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e37786-4e69-4cb4-b73b-b88b3b4f429c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ff523319eb74ebf8b298d9a8b3e79cd)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:28.791361 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e37786-4e69-4cb4-b73b-b88b3b4f429c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:28.829412 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 921/1850] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:05:28 2022] POST /identity/v3/projects => generated 385 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:05:28.834675 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3035fb-34ed-4bcd-914f-03665e4115e4 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:28.837191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-cf3035fb-34ed-4bcd-914f-03665e4115e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:28.866398 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-a6e37786-4e69-4cb4-b73b-b88b3b4f429c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ff523319eb74ebf8b298d9a8b3e79cd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:28.867796 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-a6e37786-4e69-4cb4-b73b-b88b3b4f429c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ff523319eb74ebf8b298d9a8b3e79cd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:28.878291 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-a6e37786-4e69-4cb4-b73b-b88b3b4f429c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ff523319eb74ebf8b298d9a8b3e79cd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:28.879331 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-a6e37786-4e69-4cb4-b73b-b88b3b4f429c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ff523319eb74ebf8b298d9a8b3e79cd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:28.890436 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-a6e37786-4e69-4cb4-b73b-b88b3b4f429c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ff523319eb74ebf8b298d9a8b3e79cd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:28.934505 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 930/1851] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:28 2022] DELETE /identity/v3/users/2ff523319eb74ebf8b298d9a8b3e79cd => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:05:28.957277 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3035fb-34ed-4bcd-914f-03665e4115e4 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:29.035198 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e37786-4e69-4cb4-b73b-b88b3b4f429c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:29.037479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a6e37786-4e69-4cb4-b73b-b88b3b4f429c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:29.077928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaefba1-5127-4489-abe7-fdebe1a3d698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:29.078546 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaefba1-5127-4489-abe7-fdebe1a3d698 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:29.078836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaefba1-5127-4489-abe7-fdebe1a3d698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:29.079112 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaefba1-5127-4489-abe7-fdebe1a3d698 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:29.079788 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaefba1-5127-4489-abe7-fdebe1a3d698 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:29.081733 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaefba1-5127-4489-abe7-fdebe1a3d698 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:29.138244 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 922/1852] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:05:28 2022] POST /identity/v3/users => generated 293 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:29.140391 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaefba1-5127-4489-abe7-fdebe1a3d698 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:29.144559 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5eaefba1-5127-4489-abe7-fdebe1a3d698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:29.162211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaefba1-5127-4489-abe7-fdebe1a3d698 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:29.168955 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7ec46-24bd-4cca-b9e3-f2f85ea6dff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:29.169592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7ec46-24bd-4cca-b9e3-f2f85ea6dff8 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:29.169875 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7ec46-24bd-4cca-b9e3-f2f85ea6dff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:29.170154 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7ec46-24bd-4cca-b9e3-f2f85ea6dff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:29.170942 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7ec46-24bd-4cca-b9e3-f2f85ea6dff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:29.173119 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7ec46-24bd-4cca-b9e3-f2f85ea6dff8 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:29.185579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 923/1853] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:29.190299 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7ec46-24bd-4cca-b9e3-f2f85ea6dff8 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:29.195496 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:29.207479 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7ec46-24bd-4cca-b9e3-f2f85ea6dff8 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:29.208674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e37786-4e69-4cb4-b73b-b88b3b4f429c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:29.213520 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e733a1-1129-45b4-9fb6-573915ef3900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:29.214342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e733a1-1129-45b4-9fb6-573915ef3900 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:29.214801 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e733a1-1129-45b4-9fb6-573915ef3900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:29.215225 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e733a1-1129-45b4-9fb6-573915ef3900 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:29.216139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e733a1-1129-45b4-9fb6-573915ef3900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:29.217300 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d34e4-b2e9-420d-9f88-e47da0e3c491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:29.218124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d34e4-b2e9-420d-9f88-e47da0e3c491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:29.218309 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e733a1-1129-45b4-9fb6-573915ef3900 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:29.219055 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d34e4-b2e9-420d-9f88-e47da0e3c491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:29.219387 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d34e4-b2e9-420d-9f88-e47da0e3c491 admin admin] PATH_INFO: `/v3/projects/4a3bca454f1f4946a2edf357b21530a2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:29.220903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d34e4-b2e9-420d-9f88-e47da0e3c491 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a3bca454f1f4946a2edf357b21530a2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:29.224761 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d34e4-b2e9-420d-9f88-e47da0e3c491 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:29.232487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 924/1854] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:29.236223 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e733a1-1129-45b4-9fb6-573915ef3900 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:29.249094 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e733a1-1129-45b4-9fb6-573915ef3900 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:29.254749 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5eb6c-50cf-43d5-9044-a7af671422a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:29.256437 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5eb6c-50cf-43d5-9044-a7af671422a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:29.257205 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5eb6c-50cf-43d5-9044-a7af671422a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:29.257835 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5eb6c-50cf-43d5-9044-a7af671422a7 admin admin] PATH_INFO: `/v3/projects/3cd778ba2cd24d54ba42007c51c30cd8/users/bb7595aed0a942539c906937b0543135/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:29.276112 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:29.280421 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b5eb6c-50cf-43d5-9044-a7af671422a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cd778ba2cd24d54ba42007c51c30cd8, user_id=bb7595aed0a942539c906937b0543135, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:29.284233 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b5eb6c-50cf-43d5-9044-a7af671422a7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:29.307608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 925/1855] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:29 2022] PUT /identity/v3/projects/3cd778ba2cd24d54ba42007c51c30cd8/users/bb7595aed0a942539c906937b0543135/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:29.317006 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d81131-b025-4a1b-b897-e4858ddd1265 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:29.317816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d81131-b025-4a1b-b897-e4858ddd1265 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:29.318945 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f86d34e4-b2e9-420d-9f88-e47da0e3c491 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a3bca454f1f4946a2edf357b21530a2', 'request_id': 'req-f86d34e4-b2e9-420d-9f88-e47da0e3c491'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:29.319081 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d81131-b025-4a1b-b897-e4858ddd1265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:29.329209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f86d34e4-b2e9-420d-9f88-e47da0e3c491 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a3bca454f1f4946a2edf357b21530a2', 'request_id': 'req-f86d34e4-b2e9-420d-9f88-e47da0e3c491'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:29.351891 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f86d34e4-b2e9-420d-9f88-e47da0e3c491 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a3bca454f1f4946a2edf357b21530a2', 'request_id': 'req-f86d34e4-b2e9-420d-9f88-e47da0e3c491'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:29.365415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-57d81131-b025-4a1b-b897-e4858ddd1265 None None] MFA Rules not processed for user `bb7595aed0a942539c906937b0543135`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:29.367842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-57d81131-b025-4a1b-b897-e4858ddd1265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:29.368745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f86d34e4-b2e9-420d-9f88-e47da0e3c491 admin admin] The token cache is being invalidate because project 4a3bca454f1f4946a2edf357b21530a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:29.369422 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f86d34e4-b2e9-420d-9f88-e47da0e3c491 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:29.370278 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f86d34e4-b2e9-420d-9f88-e47da0e3c491 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:29.373430 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 931/1856] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:29 2022] DELETE /identity/v3/projects/4a3bca454f1f4946a2edf357b21530a2 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:29.377074 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d34e4-b2e9-420d-9f88-e47da0e3c491 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:29.379135 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f86d34e4-b2e9-420d-9f88-e47da0e3c491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:29.502015 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 926/1857] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:05:29.503709 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d81131-b025-4a1b-b897-e4858ddd1265 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:29.505578 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-57d81131-b025-4a1b-b897-e4858ddd1265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:29.521364 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d81131-b025-4a1b-b897-e4858ddd1265 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:29.524577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d34e4-b2e9-420d-9f88-e47da0e3c491 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:29.533111 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5e399-5141-43fc-a8ca-aea723f4bf6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:29.533861 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5e399-5141-43fc-a8ca-aea723f4bf6e admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:29.534237 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5e399-5141-43fc-a8ca-aea723f4bf6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:29.534704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5e399-5141-43fc-a8ca-aea723f4bf6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:29.535954 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f5e399-5141-43fc-a8ca-aea723f4bf6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:29.538020 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f65041-ace8-403f-9151-f7e3c51032ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:29.538465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f5e399-5141-43fc-a8ca-aea723f4bf6e admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:29.538683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f65041-ace8-403f-9151-f7e3c51032ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:29.538965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f65041-ace8-403f-9151-f7e3c51032ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:29.539238 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f65041-ace8-403f-9151-f7e3c51032ab admin admin] PATH_INFO: `/v3/users/e2a1a9731319485885db076102025e46` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:29.544359 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f65041-ace8-403f-9151-f7e3c51032ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2a1a9731319485885db076102025e46)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:29.546349 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f65041-ace8-403f-9151-f7e3c51032ab admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:29.585431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 927/1858] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:05:29 2022] POST /identity/v3/projects => generated 383 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:29.589743 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5e399-5141-43fc-a8ca-aea723f4bf6e admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:29.613445 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-46f65041-ace8-403f-9151-f7e3c51032ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2a1a9731319485885db076102025e46'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:29.614312 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-46f65041-ace8-403f-9151-f7e3c51032ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2a1a9731319485885db076102025e46'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:29.625155 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-46f65041-ace8-403f-9151-f7e3c51032ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2a1a9731319485885db076102025e46'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:29.625834 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-46f65041-ace8-403f-9151-f7e3c51032ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2a1a9731319485885db076102025e46'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:29.640049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-46f65041-ace8-403f-9151-f7e3c51032ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2a1a9731319485885db076102025e46'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:29.661508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 932/1859] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:29 2022] DELETE /identity/v3/users/e2a1a9731319485885db076102025e46 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:29.662913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f65041-ace8-403f-9151-f7e3c51032ab admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:29.681058 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-46f65041-ace8-403f-9151-f7e3c51032ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:29.689693 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:29.701406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5e399-5141-43fc-a8ca-aea723f4bf6e admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:29.778516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87173b41-0e8e-4220-9e24-1905010eb966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:29.779159 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87173b41-0e8e-4220-9e24-1905010eb966 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:29.779414 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87173b41-0e8e-4220-9e24-1905010eb966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:29.779730 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87173b41-0e8e-4220-9e24-1905010eb966 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:29.780509 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87173b41-0e8e-4220-9e24-1905010eb966 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:29.783505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87173b41-0e8e-4220-9e24-1905010eb966 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:29.791019 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f65041-ace8-403f-9151-f7e3c51032ab admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:29.795771 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:29.796601 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbe504-f04c-42d5-bbe8-87ed4b366fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:29.797389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbe504-f04c-42d5-bbe8-87ed4b366fc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:29.797791 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbe504-f04c-42d5-bbe8-87ed4b366fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:29.798236 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbe504-f04c-42d5-bbe8-87ed4b366fc8 admin admin] PATH_INFO: `/v3/projects/b08679cb668c4533b886b66f4fd20fde` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:29.799646 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbe504-f04c-42d5-bbe8-87ed4b366fc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b08679cb668c4533b886b66f4fd20fde)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:29.801901 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbe504-f04c-42d5-bbe8-87ed4b366fc8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:29.810730 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 928/1860] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:05:29 2022] POST /identity/v3/users => generated 298 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:29.814547 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87173b41-0e8e-4220-9e24-1905010eb966 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:29.816372 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-87173b41-0e8e-4220-9e24-1905010eb966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:29.832535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87173b41-0e8e-4220-9e24-1905010eb966 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:29.840503 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7cbfa-912f-420c-a312-11ef2101659a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:29.840503 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7cbfa-912f-420c-a312-11ef2101659a admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:29.840745 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7cbfa-912f-420c-a312-11ef2101659a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:29.840884 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7cbfa-912f-420c-a312-11ef2101659a admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:29.841536 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e7cbfa-912f-420c-a312-11ef2101659a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:29.843221 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e7cbfa-912f-420c-a312-11ef2101659a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:29.860342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 929/1861] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:05:29.864267 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7cbfa-912f-420c-a312-11ef2101659a admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:29.866483 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c4dbe504-f04c-42d5-bbe8-87ed4b366fc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b08679cb668c4533b886b66f4fd20fde', 'request_id': 'req-c4dbe504-f04c-42d5-bbe8-87ed4b366fc8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:29.876415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7cbfa-912f-420c-a312-11ef2101659a admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:29.877246 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c4dbe504-f04c-42d5-bbe8-87ed4b366fc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b08679cb668c4533b886b66f4fd20fde', 'request_id': 'req-c4dbe504-f04c-42d5-bbe8-87ed4b366fc8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:29.892537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c4dbe504-f04c-42d5-bbe8-87ed4b366fc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b08679cb668c4533b886b66f4fd20fde', 'request_id': 'req-c4dbe504-f04c-42d5-bbe8-87ed4b366fc8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:29.908259 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c4dbe504-f04c-42d5-bbe8-87ed4b366fc8 admin admin] The token cache is being invalidate because project b08679cb668c4533b886b66f4fd20fde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:29.908521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c4dbe504-f04c-42d5-bbe8-87ed4b366fc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:29.909220 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c4dbe504-f04c-42d5-bbe8-87ed4b366fc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:29.912204 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 933/1862] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:29 2022] DELETE /identity/v3/projects/b08679cb668c4533b886b66f4fd20fde => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:29.921264 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358148d1-9fbf-4052-b2b7-92435374388a None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:29.921565 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358148d1-9fbf-4052-b2b7-92435374388a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:29.921842 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358148d1-9fbf-4052-b2b7-92435374388a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:29.966079 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f9e317-5f0e-4a2a-a900-089ebe084ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:29.966833 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f9e317-5f0e-4a2a-a900-089ebe084ba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:29.967191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f9e317-5f0e-4a2a-a900-089ebe084ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:29.967534 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f9e317-5f0e-4a2a-a900-089ebe084ba5 admin admin] PATH_INFO: `/v3/projects/81c7ee8953e54b0fbfb16e612573f832/users/0da396e6cebc4abdab4f96b4f09d6525/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:29.978628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-358148d1-9fbf-4052-b2b7-92435374388a None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:29.980653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-358148d1-9fbf-4052-b2b7-92435374388a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:29.990716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f9e317-5f0e-4a2a-a900-089ebe084ba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81c7ee8953e54b0fbfb16e612573f832, user_id=0da396e6cebc4abdab4f96b4f09d6525, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:29.998093 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f9e317-5f0e-4a2a-a900-089ebe084ba5 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:29.998265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 934/1863] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:30.000688 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9bc48f-f472-4a8f-8595-b2b80c2c03af None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:30.001329 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9bc48f-f472-4a8f-8595-b2b80c2c03af None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:30.001816 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9bc48f-f472-4a8f-8595-b2b80c2c03af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:30.029793 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-8a9bc48f-f472-4a8f-8595-b2b80c2c03af None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:30.031914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8a9bc48f-f472-4a8f-8595-b2b80c2c03af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:30.032393 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 930/1864] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:29 2022] PUT /identity/v3/projects/81c7ee8953e54b0fbfb16e612573f832/users/0da396e6cebc4abdab4f96b4f09d6525/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:30.037318 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f9e317-5f0e-4a2a-a900-089ebe084ba5 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:30.039457 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-08f9e317-5f0e-4a2a-a900-089ebe084ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:30.166863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f9e317-5f0e-4a2a-a900-089ebe084ba5 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:30.172896 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa25422-0f53-41c3-9308-41d1236196f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:30.173296 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 935/1865] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:05:30.174575 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9bc48f-f472-4a8f-8595-b2b80c2c03af None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:30.176423 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8a9bc48f-f472-4a8f-8595-b2b80c2c03af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:30.178212 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa25422-0f53-41c3-9308-41d1236196f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:30.178658 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa25422-0f53-41c3-9308-41d1236196f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:30.178959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa25422-0f53-41c3-9308-41d1236196f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:30.179804 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa25422-0f53-41c3-9308-41d1236196f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:30.181692 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa25422-0f53-41c3-9308-41d1236196f5 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:30.209685 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 931/1866] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:30.211072 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9bc48f-f472-4a8f-8595-b2b80c2c03af None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:30.213315 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa25422-0f53-41c3-9308-41d1236196f5 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:30.218679 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52317ceb-bb7e-4bf7-82b4-0613517d01b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:30.219714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52317ceb-bb7e-4bf7-82b4-0613517d01b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:30.220277 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52317ceb-bb7e-4bf7-82b4-0613517d01b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:30.221527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52317ceb-bb7e-4bf7-82b4-0613517d01b6 admin admin] PATH_INFO: `/v3/projects/32c9b9e47a43415aa9cc2b083c1ad8e8` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:30.223239 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52317ceb-bb7e-4bf7-82b4-0613517d01b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32c9b9e47a43415aa9cc2b083c1ad8e8)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:30.226338 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52317ceb-bb7e-4bf7-82b4-0613517d01b6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:30.228205 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa25422-0f53-41c3-9308-41d1236196f5 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:30.239419 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd0f85-582d-42c5-93b3-fdec62af439b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:30.242060 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd0f85-582d-42c5-93b3-fdec62af439b admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:30.242060 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd0f85-582d-42c5-93b3-fdec62af439b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:30.242060 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd0f85-582d-42c5-93b3-fdec62af439b admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:30.242533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd0f85-582d-42c5-93b3-fdec62af439b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:05:30.243264 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd0f85-582d-42c5-93b3-fdec62af439b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:30.245096 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd0f85-582d-42c5-93b3-fdec62af439b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:30.268862 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 932/1867] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:05:30.268862 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd0f85-582d-42c5-93b3-fdec62af439b admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:30.275133 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:30.285987 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd0f85-582d-42c5-93b3-fdec62af439b admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:30.292608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842518e8-e7ba-4ed7-b7f0-00add0d9feec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:30.293490 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842518e8-e7ba-4ed7-b7f0-00add0d9feec admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:30.294678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842518e8-e7ba-4ed7-b7f0-00add0d9feec admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:30.294678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842518e8-e7ba-4ed7-b7f0-00add0d9feec admin admin] PATH_INFO: `/v3/projects/81c7ee8953e54b0fbfb16e612573f832/users/0da396e6cebc4abdab4f96b4f09d6525/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:30.298078 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842518e8-e7ba-4ed7-b7f0-00add0d9feec admin admin] RBAC: Authorizing `identity:create_grant(project_id=81c7ee8953e54b0fbfb16e612573f832, user_id=0da396e6cebc4abdab4f96b4f09d6525, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:30.303084 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842518e8-e7ba-4ed7-b7f0-00add0d9feec admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:30.315047 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-52317ceb-bb7e-4bf7-82b4-0613517d01b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32c9b9e47a43415aa9cc2b083c1ad8e8', 'request_id': 'req-52317ceb-bb7e-4bf7-82b4-0613517d01b6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:30.317546 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 933/1868] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:30 2022] PUT /identity/v3/projects/81c7ee8953e54b0fbfb16e612573f832/users/0da396e6cebc4abdab4f96b4f09d6525/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:30.319328 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842518e8-e7ba-4ed7-b7f0-00add0d9feec admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:30.332943 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-52317ceb-bb7e-4bf7-82b4-0613517d01b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32c9b9e47a43415aa9cc2b083c1ad8e8', 'request_id': 'req-52317ceb-bb7e-4bf7-82b4-0613517d01b6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:30.341927 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-52317ceb-bb7e-4bf7-82b4-0613517d01b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32c9b9e47a43415aa9cc2b083c1ad8e8', 'request_id': 'req-52317ceb-bb7e-4bf7-82b4-0613517d01b6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:30.356650 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-52317ceb-bb7e-4bf7-82b4-0613517d01b6 admin admin] The token cache is being invalidate because project 32c9b9e47a43415aa9cc2b083c1ad8e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:30.357164 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-52317ceb-bb7e-4bf7-82b4-0613517d01b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:30.358009 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-52317ceb-bb7e-4bf7-82b4-0613517d01b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:30.361400 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 936/1869] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:30 2022] DELETE /identity/v3/projects/32c9b9e47a43415aa9cc2b083c1ad8e8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:30.362798 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45023d-2cad-47db-9b02-afb89c221159 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:30.363823 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45023d-2cad-47db-9b02-afb89c221159 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:30.365499 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45023d-2cad-47db-9b02-afb89c221159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:30.393579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:30.406962 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4b45023d-2cad-47db-9b02-afb89c221159 None None] MFA Rules not processed for user `0da396e6cebc4abdab4f96b4f09d6525`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:30.408535 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4b45023d-2cad-47db-9b02-afb89c221159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:30.425364 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842518e8-e7ba-4ed7-b7f0-00add0d9feec admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:30.431882 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed4ca6-a53a-41d5-a582-a40dac7ce891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:30.432493 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed4ca6-a53a-41d5-a582-a40dac7ce891 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:30.432777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed4ca6-a53a-41d5-a582-a40dac7ce891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:30.433184 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed4ca6-a53a-41d5-a582-a40dac7ce891 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:30.433827 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baed4ca6-a53a-41d5-a582-a40dac7ce891 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:30.435762 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baed4ca6-a53a-41d5-a582-a40dac7ce891 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:30.471219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 934/1870] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:05:30 2022] POST /identity/v3/projects => generated 391 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:30.471219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0c3a7-1886-4f47-ada7-3ff16d9de22f None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:30.471219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0c3a7-1886-4f47-ada7-3ff16d9de22f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:30.471219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0c3a7-1886-4f47-ada7-3ff16d9de22f None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:30.473378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 935/1871] 10.222.0.42 () {54 vars in 921 bytes} [Wed Sep 14 01:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:05:30.473378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0c3a7-1886-4f47-ada7-3ff16d9de22f None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:30.475114 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7ad0c3a7-1886-4f47-ada7-3ff16d9de22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:30.520550 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 937/1872] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:30 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:30.521313 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45023d-2cad-47db-9b02-afb89c221159 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:30.528042 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4b45023d-2cad-47db-9b02-afb89c221159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:30.547258 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45023d-2cad-47db-9b02-afb89c221159 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:30.582381 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0c3a7-1886-4f47-ada7-3ff16d9de22f None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:30.589647 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9238d-28b4-43b2-933f-cc4d8a7ed2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:30.590304 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9238d-28b4-43b2-933f-cc4d8a7ed2ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:30.590593 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9238d-28b4-43b2-933f-cc4d8a7ed2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:30.590890 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9238d-28b4-43b2-933f-cc4d8a7ed2ef admin admin] PATH_INFO: `/v3/users/b33eff288a194ccbaf8aa7aeb9ceddd1` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:30.593396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9238d-28b4-43b2-933f-cc4d8a7ed2ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=b33eff288a194ccbaf8aa7aeb9ceddd1)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:30.596170 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9238d-28b4-43b2-933f-cc4d8a7ed2ef admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:30.626667 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc458b-7f16-4ab4-a1e0-8c25362602cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:30.628234 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc458b-7f16-4ab4-a1e0-8c25362602cb admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:30.629069 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc458b-7f16-4ab4-a1e0-8c25362602cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:30.629581 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc458b-7f16-4ab4-a1e0-8c25362602cb admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:30.630671 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc458b-7f16-4ab4-a1e0-8c25362602cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:30.633183 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cc458b-7f16-4ab4-a1e0-8c25362602cb admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:30.671752 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 938/1873] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:05:30 2022] POST /identity/v3/users => generated 296 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:30.675267 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2eb9238d-28b4-43b2-933f-cc4d8a7ed2ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b33eff288a194ccbaf8aa7aeb9ceddd1'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:30.676172 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2eb9238d-28b4-43b2-933f-cc4d8a7ed2ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b33eff288a194ccbaf8aa7aeb9ceddd1'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:30.679467 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699584e-c608-4995-86ae-23bcddcb5a23 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:30.679925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699584e-c608-4995-86ae-23bcddcb5a23 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:30.680207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699584e-c608-4995-86ae-23bcddcb5a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:30.687211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2eb9238d-28b4-43b2-933f-cc4d8a7ed2ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b33eff288a194ccbaf8aa7aeb9ceddd1'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:30.716291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2eb9238d-28b4-43b2-933f-cc4d8a7ed2ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b33eff288a194ccbaf8aa7aeb9ceddd1'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:30.717092 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2eb9238d-28b4-43b2-933f-cc4d8a7ed2ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b33eff288a194ccbaf8aa7aeb9ceddd1'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:30.719006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-5699584e-c608-4995-86ae-23bcddcb5a23 None None] MFA Rules not processed for user `5e0fe2e84f6748c992c9beedf2f5d49b`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:30.720724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5699584e-c608-4995-86ae-23bcddcb5a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:30.746670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 936/1874] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:30 2022] DELETE /identity/v3/users/b33eff288a194ccbaf8aa7aeb9ceddd1 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:30.752644 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9238d-28b4-43b2-933f-cc4d8a7ed2ef admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:30.756697 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2eb9238d-28b4-43b2-933f-cc4d8a7ed2ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:30.761450 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:30.820161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:30.837620 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 939/1875] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Sep 14 01:05:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:30.838651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699584e-c608-4995-86ae-23bcddcb5a23 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:30.840349 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5699584e-c608-4995-86ae-23bcddcb5a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:30.843030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9238d-28b4-43b2-933f-cc4d8a7ed2ef admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:30.851400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48efe6ab-8fad-4f75-a31c-97702fcf11dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:30.852763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48efe6ab-8fad-4f75-a31c-97702fcf11dd service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:30.852886 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48efe6ab-8fad-4f75-a31c-97702fcf11dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:30.853167 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48efe6ab-8fad-4f75-a31c-97702fcf11dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:30.857185 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-48efe6ab-8fad-4f75-a31c-97702fcf11dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:30.864176 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699584e-c608-4995-86ae-23bcddcb5a23 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:30.950526 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd51f02-2b9b-4bde-8ad9-9e52f3e61a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:30.951427 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd51f02-2b9b-4bde-8ad9-9e52f3e61a72 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:30.951773 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd51f02-2b9b-4bde-8ad9-9e52f3e61a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:30.952097 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd51f02-2b9b-4bde-8ad9-9e52f3e61a72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:30.952813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd51f02-2b9b-4bde-8ad9-9e52f3e61a72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:30.954449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd51f02-2b9b-4bde-8ad9-9e52f3e61a72 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:30.972513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 940/1876] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:30.973785 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd51f02-2b9b-4bde-8ad9-9e52f3e61a72 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:30.976264 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2dd51f02-2b9b-4bde-8ad9-9e52f3e61a72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:30.995028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48efe6ab-8fad-4f75-a31c-97702fcf11dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:30.999301 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48efe6ab-8fad-4f75-a31c-97702fcf11dd service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:31.032223 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 937/1877] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:31.033246 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48efe6ab-8fad-4f75-a31c-97702fcf11dd service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:31.057371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48efe6ab-8fad-4f75-a31c-97702fcf11dd service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:31.063922 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff37ee9-16e2-4374-b1ed-33bf660241b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:31.064818 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff37ee9-16e2-4374-b1ed-33bf660241b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:31.065362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff37ee9-16e2-4374-b1ed-33bf660241b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:31.065793 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff37ee9-16e2-4374-b1ed-33bf660241b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:31.066625 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff37ee9-16e2-4374-b1ed-33bf660241b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:31.069834 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff37ee9-16e2-4374-b1ed-33bf660241b7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:31.081549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd51f02-2b9b-4bde-8ad9-9e52f3e61a72 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:31.082644 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 938/1878] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:31.087089 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff37ee9-16e2-4374-b1ed-33bf660241b7 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:31.089599 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8389f7-7c1e-4a3b-9862-eccf0cd4fc5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:31.090513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8389f7-7c1e-4a3b-9862-eccf0cd4fc5c service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:31.091153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8389f7-7c1e-4a3b-9862-eccf0cd4fc5c service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:31.091411 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8389f7-7c1e-4a3b-9862-eccf0cd4fc5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:31.104779 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8389f7-7c1e-4a3b-9862-eccf0cd4fc5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:31.105153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff37ee9-16e2-4374-b1ed-33bf660241b7 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:31.106634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8389f7-7c1e-4a3b-9862-eccf0cd4fc5c service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:31.114238 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949b904-8e8f-4c8b-9e27-3dddc6b90b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:31.116267 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949b904-8e8f-4c8b-9e27-3dddc6b90b27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:31.117549 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949b904-8e8f-4c8b-9e27-3dddc6b90b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:31.117995 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949b904-8e8f-4c8b-9e27-3dddc6b90b27 admin admin] PATH_INFO: `/v3/projects/f2194a8dfe9b4af4b6fbb1fd1c136f99/users/932bdd0285cc42cba711420d0860dfb0/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:31.121142 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 941/1879] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:31.167458 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2949b904-8e8f-4c8b-9e27-3dddc6b90b27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2194a8dfe9b4af4b6fbb1fd1c136f99, user_id=932bdd0285cc42cba711420d0860dfb0, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:31.170976 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2949b904-8e8f-4c8b-9e27-3dddc6b90b27 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:31.186430 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8389f7-7c1e-4a3b-9862-eccf0cd4fc5c service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:31.187986 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bc8389f7-7c1e-4a3b-9862-eccf0cd4fc5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:31.191501 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 939/1880] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:31 2022] PUT /identity/v3/projects/f2194a8dfe9b4af4b6fbb1fd1c136f99/users/932bdd0285cc42cba711420d0860dfb0/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:31.212109 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c836107-a81b-4b62-bd33-05c2222dba10 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:31.212616 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c836107-a81b-4b62-bd33-05c2222dba10 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:31.213751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c836107-a81b-4b62-bd33-05c2222dba10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:31.220145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8389f7-7c1e-4a3b-9862-eccf0cd4fc5c service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:31.271415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-5c836107-a81b-4b62-bd33-05c2222dba10 None None] MFA Rules not processed for user `932bdd0285cc42cba711420d0860dfb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:31.273136 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5c836107-a81b-4b62-bd33-05c2222dba10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:31.287376 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:31.333381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4047e98-575f-4113-9220-cf1e10907bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:31.334435 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4047e98-575f-4113-9220-cf1e10907bb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:31.334724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4047e98-575f-4113-9220-cf1e10907bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:31.335005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4047e98-575f-4113-9220-cf1e10907bb9 admin admin] PATH_INFO: `/v3/projects/d8ee4628e9f349369ea76d4e9d399733` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:31.337167 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4047e98-575f-4113-9220-cf1e10907bb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8ee4628e9f349369ea76d4e9d399733)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:31.339481 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4047e98-575f-4113-9220-cf1e10907bb9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:31.380627 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 940/1881] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:31 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:31.385224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c836107-a81b-4b62-bd33-05c2222dba10 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:31.398663 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c836107-a81b-4b62-bd33-05c2222dba10 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:31.424669 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d4047e98-575f-4113-9220-cf1e10907bb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8ee4628e9f349369ea76d4e9d399733', 'request_id': 'req-d4047e98-575f-4113-9220-cf1e10907bb9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:31.435955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d4047e98-575f-4113-9220-cf1e10907bb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8ee4628e9f349369ea76d4e9d399733', 'request_id': 'req-d4047e98-575f-4113-9220-cf1e10907bb9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:31.443175 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d4047e98-575f-4113-9220-cf1e10907bb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8ee4628e9f349369ea76d4e9d399733', 'request_id': 'req-d4047e98-575f-4113-9220-cf1e10907bb9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:31.457031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d4047e98-575f-4113-9220-cf1e10907bb9 admin admin] The token cache is being invalidate because project d8ee4628e9f349369ea76d4e9d399733 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:31.458056 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d4047e98-575f-4113-9220-cf1e10907bb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:31.459879 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d4047e98-575f-4113-9220-cf1e10907bb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:31.463576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 942/1882] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:31 2022] DELETE /identity/v3/projects/d8ee4628e9f349369ea76d4e9d399733 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:05:31.475898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23d6ce-f24c-4332-96df-09bddf843fe5 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:31.480893 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23d6ce-f24c-4332-96df-09bddf843fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:31.481872 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23d6ce-f24c-4332-96df-09bddf843fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:31.501644 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:31.537146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df9798-8d2d-4bc0-964d-9a3ae662c71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:31.537940 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df9798-8d2d-4bc0-964d-9a3ae662c71c admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:31.539701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df9798-8d2d-4bc0-964d-9a3ae662c71c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:31.539998 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df9798-8d2d-4bc0-964d-9a3ae662c71c admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:31.540661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57df9798-8d2d-4bc0-964d-9a3ae662c71c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:31.544372 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-9d23d6ce-f24c-4332-96df-09bddf843fe5 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:31.546280 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9d23d6ce-f24c-4332-96df-09bddf843fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:31.547098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57df9798-8d2d-4bc0-964d-9a3ae662c71c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:31.557071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 943/1883] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:31.572852 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08689d24-213d-48e5-8c2b-5e6854472a2e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:31.573137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08689d24-213d-48e5-8c2b-5e6854472a2e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:31.573417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08689d24-213d-48e5-8c2b-5e6854472a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:31.601435 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 941/1884] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:05:31 2022] POST /identity/v3/projects => generated 391 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:31.606745 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df9798-8d2d-4bc0-964d-9a3ae662c71c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:31.610799 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-57df9798-8d2d-4bc0-964d-9a3ae662c71c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:31.620007 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-08689d24-213d-48e5-8c2b-5e6854472a2e None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:31.621677 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-08689d24-213d-48e5-8c2b-5e6854472a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:31.693624 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df9798-8d2d-4bc0-964d-9a3ae662c71c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:31.700627 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bea076-1258-4c60-90c4-746949c5097f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:31.701344 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bea076-1258-4c60-90c4-746949c5097f admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:31.701700 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bea076-1258-4c60-90c4-746949c5097f admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:31.702196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bea076-1258-4c60-90c4-746949c5097f admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:31.702911 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bea076-1258-4c60-90c4-746949c5097f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:31.705137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bea076-1258-4c60-90c4-746949c5097f admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:31.741459 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 944/1885] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:31.744218 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08689d24-213d-48e5-8c2b-5e6854472a2e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:31.756039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 942/1886] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:05:31 2022] POST /identity/v3/users => generated 302 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:31.757969 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bea076-1258-4c60-90c4-746949c5097f admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:31.766208 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08689d24-213d-48e5-8c2b-5e6854472a2e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:31.772148 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bea076-1258-4c60-90c4-746949c5097f admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:31.778461 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd5a00-c73d-485b-853b-7e769c8676e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:31.778783 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94925dc7-e106-41b2-8929-93244eb67722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:31.779239 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd5a00-c73d-485b-853b-7e769c8676e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:31.779606 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd5a00-c73d-485b-853b-7e769c8676e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:31.779940 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94925dc7-e106-41b2-8929-93244eb67722 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:31.780151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd5a00-c73d-485b-853b-7e769c8676e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:31.780532 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94925dc7-e106-41b2-8929-93244eb67722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:31.780967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94925dc7-e106-41b2-8929-93244eb67722 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:31.783120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94925dc7-e106-41b2-8929-93244eb67722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:31.783639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd5a00-c73d-485b-853b-7e769c8676e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:05:31.784177 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd5a00-c73d-485b-853b-7e769c8676e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:31.785315 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94925dc7-e106-41b2-8929-93244eb67722 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:31.785942 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd5a00-c73d-485b-853b-7e769c8676e7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:31.803000 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 943/1887] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:31.804493 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94925dc7-e106-41b2-8929-93244eb67722 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:31.812529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 945/1888] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:05:31.813979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd5a00-c73d-485b-853b-7e769c8676e7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:31.821160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94925dc7-e106-41b2-8929-93244eb67722 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:31.826042 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd5a00-c73d-485b-853b-7e769c8676e7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:31.827511 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2241c78-86e0-41a6-a1eb-91101fc036a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:31.828376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2241c78-86e0-41a6-a1eb-91101fc036a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:31.828802 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2241c78-86e0-41a6-a1eb-91101fc036a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:31.829296 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2241c78-86e0-41a6-a1eb-91101fc036a7 admin admin] PATH_INFO: `/v3/projects/97cbd90ce26e4e37b8f1976a08073e93/users/b49aa723a96748a7b7192eb9052b40d1/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:31.831531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c9ed0-1250-4e4e-824c-8b8301cb9f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:31.832257 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c9ed0-1250-4e4e-824c-8b8301cb9f29 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:31.832605 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c9ed0-1250-4e4e-824c-8b8301cb9f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:31.832947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c9ed0-1250-4e4e-824c-8b8301cb9f29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:31.834194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c9ed0-1250-4e4e-824c-8b8301cb9f29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:31.836373 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c9ed0-1250-4e4e-824c-8b8301cb9f29 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:31.847325 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2241c78-86e0-41a6-a1eb-91101fc036a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97cbd90ce26e4e37b8f1976a08073e93, user_id=b49aa723a96748a7b7192eb9052b40d1, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:31.850722 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2241c78-86e0-41a6-a1eb-91101fc036a7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:31.872443 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 946/1890] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:05:31 2022] POST /identity/v3/projects => generated 367 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:05:31.875783 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 944/1890] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:31 2022] PUT /identity/v3/projects/97cbd90ce26e4e37b8f1976a08073e93/users/b49aa723a96748a7b7192eb9052b40d1/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:31.878519 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2241c78-86e0-41a6-a1eb-91101fc036a7 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:31.879185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c9ed0-1250-4e4e-824c-8b8301cb9f29 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:31.932088 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:31.981053 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c9ed0-1250-4e4e-824c-8b8301cb9f29 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:31.986709 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ac628-be8c-4fb0-aae5-954e48e0495b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:31.987470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ac628-be8c-4fb0-aae5-954e48e0495b admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:31.987871 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ac628-be8c-4fb0-aae5-954e48e0495b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:31.988223 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ac628-be8c-4fb0-aae5-954e48e0495b admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:31.988949 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ac628-be8c-4fb0-aae5-954e48e0495b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:31.990838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ac628-be8c-4fb0-aae5-954e48e0495b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:32.002530 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 947/1891] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:32.004091 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2241c78-86e0-41a6-a1eb-91101fc036a7 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:32.007449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ac628-be8c-4fb0-aae5-954e48e0495b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:32.014923 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f677aa-5ff9-44a9-9e61-9a2cb70151cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:32.015706 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f677aa-5ff9-44a9-9e61-9a2cb70151cb admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:32.016064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f677aa-5ff9-44a9-9e61-9a2cb70151cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:32.016457 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f677aa-5ff9-44a9-9e61-9a2cb70151cb admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:32.018132 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f677aa-5ff9-44a9-9e61-9a2cb70151cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:32.020333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f677aa-5ff9-44a9-9e61-9a2cb70151cb admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:32.032368 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ac628-be8c-4fb0-aae5-954e48e0495b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:32.042674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c077a-ea38-4a7d-ae15-63f560e83006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:32.043387 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c077a-ea38-4a7d-ae15-63f560e83006 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:32.043664 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c077a-ea38-4a7d-ae15-63f560e83006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:32.043970 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c077a-ea38-4a7d-ae15-63f560e83006 admin admin] PATH_INFO: `/v3/projects/97cbd90ce26e4e37b8f1976a08073e93/users/b49aa723a96748a7b7192eb9052b40d1/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:32.048376 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c077a-ea38-4a7d-ae15-63f560e83006 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97cbd90ce26e4e37b8f1976a08073e93, user_id=b49aa723a96748a7b7192eb9052b40d1, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:32.051065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c077a-ea38-4a7d-ae15-63f560e83006 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:32.052491 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 945/1892] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:05:31 2022] POST /identity/v3/users => generated 284 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:32.057139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f677aa-5ff9-44a9-9e61-9a2cb70151cb admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:32.072809 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f677aa-5ff9-44a9-9e61-9a2cb70151cb admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:32.074536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 948/1893] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:32 2022] PUT /identity/v3/projects/97cbd90ce26e4e37b8f1976a08073e93/users/b49aa723a96748a7b7192eb9052b40d1/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:32.089368 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44640ff1-fec4-4ba8-99b0-884444793ed3 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:32.089721 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44640ff1-fec4-4ba8-99b0-884444793ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:32.089919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44640ff1-fec4-4ba8-99b0-884444793ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:32.096325 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:32.127171 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-44640ff1-fec4-4ba8-99b0-884444793ed3 None None] MFA Rules not processed for user `b49aa723a96748a7b7192eb9052b40d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:32.128727 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-44640ff1-fec4-4ba8-99b0-884444793ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:32.150594 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a628ef-be03-4672-9008-0bf5729c9ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:32.152821 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a628ef-be03-4672-9008-0bf5729c9ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:32.153295 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a628ef-be03-4672-9008-0bf5729c9ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:32.153723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a628ef-be03-4672-9008-0bf5729c9ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:32.154544 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a628ef-be03-4672-9008-0bf5729c9ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:32.156381 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a628ef-be03-4672-9008-0bf5729c9ca6 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:32.170797 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 946/1894] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:05:32.174844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a628ef-be03-4672-9008-0bf5729c9ca6 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:32.193224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a628ef-be03-4672-9008-0bf5729c9ca6 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:32.201039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d037e0-9e31-4515-83ac-cd731a0f9693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:32.202182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d037e0-9e31-4515-83ac-cd731a0f9693 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:32.202722 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d037e0-9e31-4515-83ac-cd731a0f9693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:32.203241 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d037e0-9e31-4515-83ac-cd731a0f9693 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:32.204101 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d037e0-9e31-4515-83ac-cd731a0f9693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:32.206079 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d037e0-9e31-4515-83ac-cd731a0f9693 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:32.249370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 947/1895] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:05:32.254094 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d037e0-9e31-4515-83ac-cd731a0f9693 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:32.279483 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d037e0-9e31-4515-83ac-cd731a0f9693 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:32.280652 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 949/1896] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:32 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:05:32.287038 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44640ff1-fec4-4ba8-99b0-884444793ed3 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:32.289114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-44640ff1-fec4-4ba8-99b0-884444793ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:32.292988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d9dbb-f2b1-4b67-9679-a5065019d132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:32.293679 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d9dbb-f2b1-4b67-9679-a5065019d132 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:32.293962 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d9dbb-f2b1-4b67-9679-a5065019d132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:32.294440 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d9dbb-f2b1-4b67-9679-a5065019d132 admin admin] PATH_INFO: `/v3/projects/a4e9e4be2bcc4ce59dd4ac80de3bdda3/users/90cfb310f700408e9d14e432426d467e/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:32.328507 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d9dbb-f2b1-4b67-9679-a5065019d132 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4e9e4be2bcc4ce59dd4ac80de3bdda3, user_id=90cfb310f700408e9d14e432426d467e, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:32.331587 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d9dbb-f2b1-4b67-9679-a5065019d132 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:32.380250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 948/1897] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:32 2022] PUT /identity/v3/projects/a4e9e4be2bcc4ce59dd4ac80de3bdda3/users/90cfb310f700408e9d14e432426d467e/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:32.384033 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4eac43-45c2-4ac1-a3e1-0c324e66e50c None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:32.384502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4eac43-45c2-4ac1-a3e1-0c324e66e50c None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:32.384933 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4eac43-45c2-4ac1-a3e1-0c324e66e50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:32.448676 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44640ff1-fec4-4ba8-99b0-884444793ed3 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:32.456615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-7d4eac43-45c2-4ac1-a3e1-0c324e66e50c None None] MFA Rules not processed for user `90cfb310f700408e9d14e432426d467e`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:32.459225 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7d4eac43-45c2-4ac1-a3e1-0c324e66e50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:32.551112 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec4e92-9cee-4228-8ba1-e24e06bdd0fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:32.552731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec4e92-9cee-4228-8ba1-e24e06bdd0fd service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:32.552897 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec4e92-9cee-4228-8ba1-e24e06bdd0fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:32.553196 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec4e92-9cee-4228-8ba1-e24e06bdd0fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:32.577075 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:32.580469 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 949/1898] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:32.599338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4eac43-45c2-4ac1-a3e1-0c324e66e50c None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:32.735642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec4e92-9cee-4228-8ba1-e24e06bdd0fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:32.735642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec4e92-9cee-4228-8ba1-e24e06bdd0fd service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:32.747896 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 950/1899] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:32 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:05:32.757742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4eac43-45c2-4ac1-a3e1-0c324e66e50c None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:32.763371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a3a69-7de2-4178-85ec-86559a052aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:32.764234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a3a69-7de2-4178-85ec-86559a052aae admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:32.764889 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a3a69-7de2-4178-85ec-86559a052aae admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:32.765258 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a3a69-7de2-4178-85ec-86559a052aae admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:32.766067 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a3a69-7de2-4178-85ec-86559a052aae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:32.768147 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a3a69-7de2-4178-85ec-86559a052aae admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:32.790764 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:32.829660 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 950/1900] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:05:32 2022] POST /identity/v3/projects => generated 365 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:32.833953 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec4e92-9cee-4228-8ba1-e24e06bdd0fd service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:32.840844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a3a69-7de2-4178-85ec-86559a052aae admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:32.847983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec4e92-9cee-4228-8ba1-e24e06bdd0fd service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:32.908809 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a3a69-7de2-4178-85ec-86559a052aae admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:32.913333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc366ba6-1d00-43ff-a534-f91ccc0512ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:32.914095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc366ba6-1d00-43ff-a534-f91ccc0512ea service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:32.914381 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc366ba6-1d00-43ff-a534-f91ccc0512ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:32.914656 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc366ba6-1d00-43ff-a534-f91ccc0512ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:32.916438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-fc366ba6-1d00-43ff-a534-f91ccc0512ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:32.918108 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6010619-c51b-409f-a9a3-b8661c70abb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:32.918782 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6010619-c51b-409f-a9a3-b8661c70abb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:32.919071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6010619-c51b-409f-a9a3-b8661c70abb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:32.919502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6010619-c51b-409f-a9a3-b8661c70abb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:32.920326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6010619-c51b-409f-a9a3-b8661c70abb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:32.922748 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6010619-c51b-409f-a9a3-b8661c70abb9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:32.949435 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 951/1901] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:05:32 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:32.950670 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6010619-c51b-409f-a9a3-b8661c70abb9 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:32.952492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e6010619-c51b-409f-a9a3-b8661c70abb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:32.968290 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6010619-c51b-409f-a9a3-b8661c70abb9 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:32.972189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5fbc9-e647-4d75-b8bd-79199ad0c19c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:32.973055 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5fbc9-e647-4d75-b8bd-79199ad0c19c service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:32.973668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5fbc9-e647-4d75-b8bd-79199ad0c19c service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:32.976218 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5fbc9-e647-4d75-b8bd-79199ad0c19c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:32.976630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c2a5fbc9-e647-4d75-b8bd-79199ad0c19c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:32.986381 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc366ba6-1d00-43ff-a534-f91ccc0512ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:32.987979 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc366ba6-1d00-43ff-a534-f91ccc0512ea service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:32.998859 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 951/1902] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:32 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:32.999708 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc366ba6-1d00-43ff-a534-f91ccc0512ea service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:33.010991 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc366ba6-1d00-43ff-a534-f91ccc0512ea service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:33.016188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4e857-eeee-4d27-9837-1629d6d56e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:33.016800 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4e857-eeee-4d27-9837-1629d6d56e59 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:33.017081 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4e857-eeee-4d27-9837-1629d6d56e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:33.017359 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4e857-eeee-4d27-9837-1629d6d56e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:33.017995 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4e857-eeee-4d27-9837-1629d6d56e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:33.019888 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4e857-eeee-4d27-9837-1629d6d56e59 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:33.030928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 952/1903] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:33.035008 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4e857-eeee-4d27-9837-1629d6d56e59 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:33.042939 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5fbc9-e647-4d75-b8bd-79199ad0c19c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:33.045082 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5fbc9-e647-4d75-b8bd-79199ad0c19c service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:33.048137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4e857-eeee-4d27-9837-1629d6d56e59 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:33.054053 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699df034-5696-46f9-9533-a56f7fea0400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:33.054742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699df034-5696-46f9-9533-a56f7fea0400 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:33.055006 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699df034-5696-46f9-9533-a56f7fea0400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:33.055284 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699df034-5696-46f9-9533-a56f7fea0400 admin admin] PATH_INFO: `/v3/projects/2f2f85f939544eeead876788ac3b0663/users/097347b6e6a44160acc1d6a79798b4be/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:33.055879 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 952/1904] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:33.072729 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699df034-5696-46f9-9533-a56f7fea0400 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f2f85f939544eeead876788ac3b0663, user_id=097347b6e6a44160acc1d6a79798b4be, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:33.075154 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699df034-5696-46f9-9533-a56f7fea0400 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:33.090362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 953/1905] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:33 2022] PUT /identity/v3/projects/2f2f85f939544eeead876788ac3b0663/users/097347b6e6a44160acc1d6a79798b4be/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:05:33.094390 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5fbc9-e647-4d75-b8bd-79199ad0c19c service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:33.107417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5fbc9-e647-4d75-b8bd-79199ad0c19c service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:33.146398 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:33.186804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb68c5-4dd6-4d88-867d-1e0dfc775f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:33.187858 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb68c5-4dd6-4d88-867d-1e0dfc775f78 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:33.188139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb68c5-4dd6-4d88-867d-1e0dfc775f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:33.188519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb68c5-4dd6-4d88-867d-1e0dfc775f78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:33.189222 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb68c5-4dd6-4d88-867d-1e0dfc775f78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:33.191310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb68c5-4dd6-4d88-867d-1e0dfc775f78 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:33.208693 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 953/1906] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:33.209859 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699df034-5696-46f9-9533-a56f7fea0400 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:33.222302 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699df034-5696-46f9-9533-a56f7fea0400 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:33.228159 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196e184-cf42-4bad-84bd-f1c1fed61dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:33.228159 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196e184-cf42-4bad-84bd-f1c1fed61dc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:33.228459 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196e184-cf42-4bad-84bd-f1c1fed61dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:33.228730 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196e184-cf42-4bad-84bd-f1c1fed61dc5 admin admin] PATH_INFO: `/v3/projects/2f2f85f939544eeead876788ac3b0663/users/097347b6e6a44160acc1d6a79798b4be/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:33.231318 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196e184-cf42-4bad-84bd-f1c1fed61dc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f2f85f939544eeead876788ac3b0663, user_id=097347b6e6a44160acc1d6a79798b4be, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:33.233961 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196e184-cf42-4bad-84bd-f1c1fed61dc5 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:33.249862 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 954/1907] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:33 2022] PUT /identity/v3/projects/2f2f85f939544eeead876788ac3b0663/users/097347b6e6a44160acc1d6a79798b4be/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:33.261995 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc822df1-8499-49af-b649-36d944c0d8c2 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:33.262725 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc822df1-8499-49af-b649-36d944c0d8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:33.262725 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc822df1-8499-49af-b649-36d944c0d8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:33.317633 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-bc822df1-8499-49af-b649-36d944c0d8c2 None None] MFA Rules not processed for user `097347b6e6a44160acc1d6a79798b4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:33.320057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bc822df1-8499-49af-b649-36d944c0d8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:33.444470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 954/1908] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:33 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:33.454528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196e184-cf42-4bad-84bd-f1c1fed61dc5 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:33.566697 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196e184-cf42-4bad-84bd-f1c1fed61dc5 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:33.574396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ecc3b-0859-40ce-a7d3-fff52cfe7e47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:33.575111 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ecc3b-0859-40ce-a7d3-fff52cfe7e47 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:33.575502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ecc3b-0859-40ce-a7d3-fff52cfe7e47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:33.575953 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ecc3b-0859-40ce-a7d3-fff52cfe7e47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:33.587495 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ecc3b-0859-40ce-a7d3-fff52cfe7e47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:33.588999 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ecc3b-0859-40ce-a7d3-fff52cfe7e47 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:33.634545 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 955/1909] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:33 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:33.738274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc822df1-8499-49af-b649-36d944c0d8c2 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:33.752824 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc822df1-8499-49af-b649-36d944c0d8c2 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:33.756595 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd0797-b924-45ba-a29e-b8af2a900677 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:33.757283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd0797-b924-45ba-a29e-b8af2a900677 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:33.757645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd0797-b924-45ba-a29e-b8af2a900677 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:33.757988 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd0797-b924-45ba-a29e-b8af2a900677 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:33.822971 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd0797-b924-45ba-a29e-b8af2a900677 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:33.824653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd0797-b924-45ba-a29e-b8af2a900677 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:33.837483 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 955/1910] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:33.851569 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ecc3b-0859-40ce-a7d3-fff52cfe7e47 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:33.858553 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:33.864150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ecc3b-0859-40ce-a7d3-fff52cfe7e47 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:33.870408 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d0cae-8df1-497a-958a-f4fe6f77d59c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:33.871030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d0cae-8df1-497a-958a-f4fe6f77d59c service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:33.871316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d0cae-8df1-497a-958a-f4fe6f77d59c service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:33.871633 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d0cae-8df1-497a-958a-f4fe6f77d59c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:33.873784 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-098d0cae-8df1-497a-958a-f4fe6f77d59c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:33.965136 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098d0cae-8df1-497a-958a-f4fe6f77d59c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:33.966939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098d0cae-8df1-497a-958a-f4fe6f77d59c service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:33.981591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 956/1911] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:33 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:36.890209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd0797-b924-45ba-a29e-b8af2a900677 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:36.963999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:36.979069 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd0797-b924-45ba-a29e-b8af2a900677 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:36.984227 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49146d-201a-4897-8fef-6f7b54bbe28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:36.985049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49146d-201a-4897-8fef-6f7b54bbe28f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:36.985500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49146d-201a-4897-8fef-6f7b54bbe28f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:36.985642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49146d-201a-4897-8fef-6f7b54bbe28f admin admin] PATH_INFO: `/v3/users/932bdd0285cc42cba711420d0860dfb0` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:36.986834 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49146d-201a-4897-8fef-6f7b54bbe28f admin admin] RBAC: Authorizing `identity:delete_user(user_id=932bdd0285cc42cba711420d0860dfb0)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:36.988831 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49146d-201a-4897-8fef-6f7b54bbe28f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:37.065415 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6f49146d-201a-4897-8fef-6f7b54bbe28f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '932bdd0285cc42cba711420d0860dfb0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:37.066272 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6f49146d-201a-4897-8fef-6f7b54bbe28f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '932bdd0285cc42cba711420d0860dfb0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:37.082552 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6f49146d-201a-4897-8fef-6f7b54bbe28f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '932bdd0285cc42cba711420d0860dfb0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:37.084023 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6f49146d-201a-4897-8fef-6f7b54bbe28f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '932bdd0285cc42cba711420d0860dfb0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:37.092733 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-6f49146d-201a-4897-8fef-6f7b54bbe28f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '932bdd0285cc42cba711420d0860dfb0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:37.119238 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 956/1912] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:36 2022] DELETE /identity/v3/users/932bdd0285cc42cba711420d0860dfb0 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:37.123912 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311a1c4-030c-4d61-abca-40ae8dbe2174 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:37.124367 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311a1c4-030c-4d61-abca-40ae8dbe2174 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:37.124461 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311a1c4-030c-4d61-abca-40ae8dbe2174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:37.159472 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-2311a1c4-030c-4d61-abca-40ae8dbe2174 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:37.161939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2311a1c4-030c-4d61-abca-40ae8dbe2174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:37.311590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 957/1913] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:37.325636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49146d-201a-4897-8fef-6f7b54bbe28f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:37.327489 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6f49146d-201a-4897-8fef-6f7b54bbe28f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:37.419351 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49146d-201a-4897-8fef-6f7b54bbe28f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:37.423079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84f7a5-9217-467e-904f-679fa3182207 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:37.423807 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84f7a5-9217-467e-904f-679fa3182207 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:37.424162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84f7a5-9217-467e-904f-679fa3182207 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:37.424504 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84f7a5-9217-467e-904f-679fa3182207 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:37.435347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84f7a5-9217-467e-904f-679fa3182207 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:37.436839 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84f7a5-9217-467e-904f-679fa3182207 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:37.447461 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 957/1914] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:38.103516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311a1c4-030c-4d61-abca-40ae8dbe2174 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:38.105369 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2311a1c4-030c-4d61-abca-40ae8dbe2174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:38.119285 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311a1c4-030c-4d61-abca-40ae8dbe2174 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:38.125838 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d0d46a-fb02-4620-87c0-849f0cf52aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:38.126484 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0d46a-fb02-4620-87c0-849f0cf52aa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:38.126757 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0d46a-fb02-4620-87c0-849f0cf52aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:38.127037 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0d46a-fb02-4620-87c0-849f0cf52aa3 admin admin] PATH_INFO: `/v3/projects/f2194a8dfe9b4af4b6fbb1fd1c136f99` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:38.128213 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d0d46a-fb02-4620-87c0-849f0cf52aa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2194a8dfe9b4af4b6fbb1fd1c136f99)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:38.130398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d0d46a-fb02-4620-87c0-849f0cf52aa3 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:38.214662 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-11d0d46a-fb02-4620-87c0-849f0cf52aa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2194a8dfe9b4af4b6fbb1fd1c136f99', 'request_id': 'req-11d0d46a-fb02-4620-87c0-849f0cf52aa3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:38.226039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-11d0d46a-fb02-4620-87c0-849f0cf52aa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2194a8dfe9b4af4b6fbb1fd1c136f99', 'request_id': 'req-11d0d46a-fb02-4620-87c0-849f0cf52aa3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:38.238210 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-11d0d46a-fb02-4620-87c0-849f0cf52aa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2194a8dfe9b4af4b6fbb1fd1c136f99', 'request_id': 'req-11d0d46a-fb02-4620-87c0-849f0cf52aa3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:38.250827 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-11d0d46a-fb02-4620-87c0-849f0cf52aa3 admin admin] The token cache is being invalidate because project f2194a8dfe9b4af4b6fbb1fd1c136f99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:38.251133 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-11d0d46a-fb02-4620-87c0-849f0cf52aa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:38.251821 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-11d0d46a-fb02-4620-87c0-849f0cf52aa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:38.253665 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 958/1915] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:38 2022] DELETE /identity/v3/projects/f2194a8dfe9b4af4b6fbb1fd1c136f99 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:38.258100 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84f7a5-9217-467e-904f-679fa3182207 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:38.274225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84f7a5-9217-467e-904f-679fa3182207 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:38.332985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ac1d5-e575-4355-bd97-7820a0b77f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:38.333896 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ac1d5-e575-4355-bd97-7820a0b77f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:38.334177 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ac1d5-e575-4355-bd97-7820a0b77f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:38.334453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ac1d5-e575-4355-bd97-7820a0b77f8a admin admin] PATH_INFO: `/v3/users/b49aa723a96748a7b7192eb9052b40d1` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:38.335605 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360ac1d5-e575-4355-bd97-7820a0b77f8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b49aa723a96748a7b7192eb9052b40d1)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:38.337558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360ac1d5-e575-4355-bd97-7820a0b77f8a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:38.352017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:38.408270 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-360ac1d5-e575-4355-bd97-7820a0b77f8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b49aa723a96748a7b7192eb9052b40d1'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:38.409024 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-360ac1d5-e575-4355-bd97-7820a0b77f8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b49aa723a96748a7b7192eb9052b40d1'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:38.433991 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-360ac1d5-e575-4355-bd97-7820a0b77f8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b49aa723a96748a7b7192eb9052b40d1'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:38.439996 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-360ac1d5-e575-4355-bd97-7820a0b77f8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b49aa723a96748a7b7192eb9052b40d1'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:38.453082 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-360ac1d5-e575-4355-bd97-7820a0b77f8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b49aa723a96748a7b7192eb9052b40d1'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:38.493575 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d0d46a-fb02-4620-87c0-849f0cf52aa3 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:38.495843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-11d0d46a-fb02-4620-87c0-849f0cf52aa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:38.499235 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 958/1916] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:38 2022] DELETE /identity/v3/users/b49aa723a96748a7b7192eb9052b40d1 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:38.510053 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d0d46a-fb02-4620-87c0-849f0cf52aa3 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:38.544363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:38.602907 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf16b208-95bd-4f21-a111-92ee8d00bc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:38.603641 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16b208-95bd-4f21-a111-92ee8d00bc5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:38.603945 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16b208-95bd-4f21-a111-92ee8d00bc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:38.604216 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16b208-95bd-4f21-a111-92ee8d00bc5f admin admin] PATH_INFO: `/v3/users/84a61f841d8b48ddaecebce8469645e4` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:38.605705 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf16b208-95bd-4f21-a111-92ee8d00bc5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=84a61f841d8b48ddaecebce8469645e4)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:38.607985 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf16b208-95bd-4f21-a111-92ee8d00bc5f admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:38.679026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-cf16b208-95bd-4f21-a111-92ee8d00bc5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84a61f841d8b48ddaecebce8469645e4'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:38.680379 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-cf16b208-95bd-4f21-a111-92ee8d00bc5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84a61f841d8b48ddaecebce8469645e4'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:38.694491 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-cf16b208-95bd-4f21-a111-92ee8d00bc5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84a61f841d8b48ddaecebce8469645e4'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:38.705342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-cf16b208-95bd-4f21-a111-92ee8d00bc5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84a61f841d8b48ddaecebce8469645e4'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:38.706249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-cf16b208-95bd-4f21-a111-92ee8d00bc5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84a61f841d8b48ddaecebce8469645e4'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:38.738768 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 959/1917] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:38 2022] DELETE /identity/v3/users/84a61f841d8b48ddaecebce8469645e4 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:38.739809 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814ffb3-f9e9-46c9-8060-058ca81f7b96 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:38.740106 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814ffb3-f9e9-46c9-8060-058ca81f7b96 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:38.740388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814ffb3-f9e9-46c9-8060-058ca81f7b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:38.774911 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4814ffb3-f9e9-46c9-8060-058ca81f7b96 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:38.776425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4814ffb3-f9e9-46c9-8060-058ca81f7b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:38.871415 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 959/1918] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:05:39.002860 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf16b208-95bd-4f21-a111-92ee8d00bc5f admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:39.004853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-cf16b208-95bd-4f21-a111-92ee8d00bc5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:39.074466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf16b208-95bd-4f21-a111-92ee8d00bc5f admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:39.078395 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018fc46-d5b9-4615-943f-ad33b94334f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:39.079003 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018fc46-d5b9-4615-943f-ad33b94334f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:39.079290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018fc46-d5b9-4615-943f-ad33b94334f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:39.086372 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018fc46-d5b9-4615-943f-ad33b94334f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:39.096612 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018fc46-d5b9-4615-943f-ad33b94334f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:39.098532 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018fc46-d5b9-4615-943f-ad33b94334f5 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:39.110292 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 960/1919] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:39.501483 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814ffb3-f9e9-46c9-8060-058ca81f7b96 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:39.506761 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4814ffb3-f9e9-46c9-8060-058ca81f7b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:39.541442 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814ffb3-f9e9-46c9-8060-058ca81f7b96 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:39.547436 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018fc46-d5b9-4615-943f-ad33b94334f5 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:39.549017 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d018fc46-d5b9-4615-943f-ad33b94334f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:39.552174 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc83c5-3395-4a1a-8d1b-11fa996e4c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:39.553683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc83c5-3395-4a1a-8d1b-11fa996e4c8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:39.554419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc83c5-3395-4a1a-8d1b-11fa996e4c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:39.555100 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc83c5-3395-4a1a-8d1b-11fa996e4c8c admin admin] PATH_INFO: `/v3/projects/97cbd90ce26e4e37b8f1976a08073e93` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:39.556445 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc83c5-3395-4a1a-8d1b-11fa996e4c8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=97cbd90ce26e4e37b8f1976a08073e93)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:39.558868 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc83c5-3395-4a1a-8d1b-11fa996e4c8c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:39.582657 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018fc46-d5b9-4615-943f-ad33b94334f5 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:39.589743 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca8d1e2-5889-4559-962b-ff699aafa73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:39.591384 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8d1e2-5889-4559-962b-ff699aafa73c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:39.591770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8d1e2-5889-4559-962b-ff699aafa73c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:39.592286 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8d1e2-5889-4559-962b-ff699aafa73c admin admin] PATH_INFO: `/v3/users/90cfb310f700408e9d14e432426d467e` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:39.593638 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca8d1e2-5889-4559-962b-ff699aafa73c admin admin] RBAC: Authorizing `identity:delete_user(user_id=90cfb310f700408e9d14e432426d467e)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:39.596531 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca8d1e2-5889-4559-962b-ff699aafa73c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:39.621261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4efc83c5-3395-4a1a-8d1b-11fa996e4c8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97cbd90ce26e4e37b8f1976a08073e93', 'request_id': 'req-4efc83c5-3395-4a1a-8d1b-11fa996e4c8c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:39.636430 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4efc83c5-3395-4a1a-8d1b-11fa996e4c8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97cbd90ce26e4e37b8f1976a08073e93', 'request_id': 'req-4efc83c5-3395-4a1a-8d1b-11fa996e4c8c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:39.648426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4efc83c5-3395-4a1a-8d1b-11fa996e4c8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97cbd90ce26e4e37b8f1976a08073e93', 'request_id': 'req-4efc83c5-3395-4a1a-8d1b-11fa996e4c8c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:39.666446 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4efc83c5-3395-4a1a-8d1b-11fa996e4c8c admin admin] The token cache is being invalidate because project 97cbd90ce26e4e37b8f1976a08073e93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:39.666748 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4efc83c5-3395-4a1a-8d1b-11fa996e4c8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:39.667679 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4efc83c5-3395-4a1a-8d1b-11fa996e4c8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:39.673566 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 960/1920] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:39 2022] DELETE /identity/v3/projects/97cbd90ce26e4e37b8f1976a08073e93 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:39.675615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5bd89-b8fc-4974-ba3b-bcd4a75c9e58 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:39.675966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5bd89-b8fc-4974-ba3b-bcd4a75c9e58 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:39.676529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5bd89-b8fc-4974-ba3b-bcd4a75c9e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:39.699047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:39.704779 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2fb5bd89-b8fc-4974-ba3b-bcd4a75c9e58 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:39.706040 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2fb5bd89-b8fc-4974-ba3b-bcd4a75c9e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:39.711386 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-bca8d1e2-5889-4559-962b-ff699aafa73c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90cfb310f700408e9d14e432426d467e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:39.712447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-bca8d1e2-5889-4559-962b-ff699aafa73c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90cfb310f700408e9d14e432426d467e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:39.720440 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:39.724617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-bca8d1e2-5889-4559-962b-ff699aafa73c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90cfb310f700408e9d14e432426d467e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:39.734785 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-bca8d1e2-5889-4559-962b-ff699aafa73c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90cfb310f700408e9d14e432426d467e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:39.735935 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-bca8d1e2-5889-4559-962b-ff699aafa73c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90cfb310f700408e9d14e432426d467e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:39.767697 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 961/1921] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:39 2022] DELETE /identity/v3/users/90cfb310f700408e9d14e432426d467e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:05:39.771055 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001626c-1dba-4529-b720-6d4dd2c7a7aa None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:39.771350 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001626c-1dba-4529-b720-6d4dd2c7a7aa None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:39.771653 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001626c-1dba-4529-b720-6d4dd2c7a7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:39.798380 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e001626c-1dba-4529-b720-6d4dd2c7a7aa None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:39.800550 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e001626c-1dba-4529-b720-6d4dd2c7a7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:39.801317 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 961/1922] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:39.809277 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20095d31-19f4-4464-a941-65cbe0119813 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:39.809768 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20095d31-19f4-4464-a941-65cbe0119813 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:39.810227 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20095d31-19f4-4464-a941-65cbe0119813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:39.843278 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-20095d31-19f4-4464-a941-65cbe0119813 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:39.844689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-20095d31-19f4-4464-a941-65cbe0119813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:39.879808 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 962/1923] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:39.890596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001626c-1dba-4529-b720-6d4dd2c7a7aa None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:39.896544 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e001626c-1dba-4529-b720-6d4dd2c7a7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:39.933030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 962/1924] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:05:39.937251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20095d31-19f4-4464-a941-65cbe0119813 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:39.948025 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20095d31-19f4-4464-a941-65cbe0119813 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:39.953326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867407e2-44af-4bae-b90f-0cc73a3db313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:39.954677 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867407e2-44af-4bae-b90f-0cc73a3db313 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:39.954980 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867407e2-44af-4bae-b90f-0cc73a3db313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:39.955236 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867407e2-44af-4bae-b90f-0cc73a3db313 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:39.956786 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867407e2-44af-4bae-b90f-0cc73a3db313 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:05:39.957100 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867407e2-44af-4bae-b90f-0cc73a3db313 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:39.959540 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867407e2-44af-4bae-b90f-0cc73a3db313 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:39.972878 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001626c-1dba-4529-b720-6d4dd2c7a7aa None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:39.974400 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 963/1925] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:05:39.978031 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26b54d-c02e-41b6-aef9-c06148baebef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:39.978661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26b54d-c02e-41b6-aef9-c06148baebef service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:39.978906 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867407e2-44af-4bae-b90f-0cc73a3db313 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:39.979022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26b54d-c02e-41b6-aef9-c06148baebef service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:39.979211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26b54d-c02e-41b6-aef9-c06148baebef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:39.989449 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf26b54d-c02e-41b6-aef9-c06148baebef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:39.991275 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf26b54d-c02e-41b6-aef9-c06148baebef service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:39.991949 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867407e2-44af-4bae-b90f-0cc73a3db313 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:39.997956 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb8d33-b79d-4527-af00-78e69a3fd560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:39.998683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb8d33-b79d-4527-af00-78e69a3fd560 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:39.999058 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb8d33-b79d-4527-af00-78e69a3fd560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:39.999448 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb8d33-b79d-4527-af00-78e69a3fd560 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:40.000209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb8d33-b79d-4527-af00-78e69a3fd560 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:40.002114 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 963/1926] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:40.003087 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb8d33-b79d-4527-af00-78e69a3fd560 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:40.033865 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 964/1927] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:05:39 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:40.038460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26b54d-c02e-41b6-aef9-c06148baebef service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:40.048547 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26b54d-c02e-41b6-aef9-c06148baebef service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:40.097691 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb8d33-b79d-4527-af00-78e69a3fd560 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:40.100166 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bebb8d33-b79d-4527-af00-78e69a3fd560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:40.163351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60cbae0-9143-487f-8a2b-ae94ed90f549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:40.164313 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cbae0-9143-487f-8a2b-ae94ed90f549 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:40.164774 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cbae0-9143-487f-8a2b-ae94ed90f549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:40.165224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cbae0-9143-487f-8a2b-ae94ed90f549 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:40.166030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60cbae0-9143-487f-8a2b-ae94ed90f549 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:40.168805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60cbae0-9143-487f-8a2b-ae94ed90f549 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:40.180552 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:40.188704 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb8d33-b79d-4527-af00-78e69a3fd560 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:40.196438 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674564e8-d927-43ac-911a-004d6dd77bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:40.197283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674564e8-d927-43ac-911a-004d6dd77bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:40.197566 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674564e8-d927-43ac-911a-004d6dd77bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:40.197887 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674564e8-d927-43ac-911a-004d6dd77bba admin admin] PATH_INFO: `/v3/projects/111d427ecff1479781c0128156960807` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:40.199032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674564e8-d927-43ac-911a-004d6dd77bba admin admin] RBAC: Authorizing `identity:delete_project(project_id=111d427ecff1479781c0128156960807)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:40.201014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674564e8-d927-43ac-911a-004d6dd77bba admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:40.203747 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 964/1928] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:05:40 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:40.207687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60cbae0-9143-487f-8a2b-ae94ed90f549 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:40.223023 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60cbae0-9143-487f-8a2b-ae94ed90f549 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:40.230034 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22758f-deba-4ac4-a557-67f6510ac2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:40.230955 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22758f-deba-4ac4-a557-67f6510ac2c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:40.231295 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:40.231547 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22758f-deba-4ac4-a557-67f6510ac2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:40.231852 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22758f-deba-4ac4-a557-67f6510ac2c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:40.232975 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e22758f-deba-4ac4-a557-67f6510ac2c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:40.234846 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e22758f-deba-4ac4-a557-67f6510ac2c0 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:40.248175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 965/1929] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:40.252220 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22758f-deba-4ac4-a557-67f6510ac2c0 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:40.260819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-674564e8-d927-43ac-911a-004d6dd77bba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '111d427ecff1479781c0128156960807', 'request_id': 'req-674564e8-d927-43ac-911a-004d6dd77bba'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:40.265487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22758f-deba-4ac4-a557-67f6510ac2c0 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:40.269970 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-674564e8-d927-43ac-911a-004d6dd77bba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '111d427ecff1479781c0128156960807', 'request_id': 'req-674564e8-d927-43ac-911a-004d6dd77bba'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:40.294295 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-674564e8-d927-43ac-911a-004d6dd77bba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '111d427ecff1479781c0128156960807', 'request_id': 'req-674564e8-d927-43ac-911a-004d6dd77bba'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:40.310721 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-674564e8-d927-43ac-911a-004d6dd77bba admin admin] The token cache is being invalidate because project 111d427ecff1479781c0128156960807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:40.311099 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-674564e8-d927-43ac-911a-004d6dd77bba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:40.311956 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-674564e8-d927-43ac-911a-004d6dd77bba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:40.313861 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 965/1930] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:40 2022] DELETE /identity/v3/projects/111d427ecff1479781c0128156960807 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:40.317715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674564e8-d927-43ac-911a-004d6dd77bba admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:40.319438 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-674564e8-d927-43ac-911a-004d6dd77bba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:40.330557 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0798c9-516b-4c60-99a0-252bbe733c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:40.331456 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0798c9-516b-4c60-99a0-252bbe733c4c admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:40.331934 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0798c9-516b-4c60-99a0-252bbe733c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:40.332362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0798c9-516b-4c60-99a0-252bbe733c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:40.333306 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0798c9-516b-4c60-99a0-252bbe733c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:40.335155 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0798c9-516b-4c60-99a0-252bbe733c4c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:40.346219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 966/1931] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:05:40.351413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0798c9-516b-4c60-99a0-252bbe733c4c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:40.353260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bf0798c9-516b-4c60-99a0-252bbe733c4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:40.367352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0798c9-516b-4c60-99a0-252bbe733c4c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:40.376879 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3065704-6dd1-4af9-96c6-0cb9aae791b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:40.377823 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3065704-6dd1-4af9-96c6-0cb9aae791b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:40.378119 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3065704-6dd1-4af9-96c6-0cb9aae791b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:40.378437 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3065704-6dd1-4af9-96c6-0cb9aae791b8 admin admin] PATH_INFO: `/v3/projects/d4e64f53fb124fa6b70e35bf156e2545/users/3f20604125354fd1b4a66fcc410da5a3/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:40.397208 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3065704-6dd1-4af9-96c6-0cb9aae791b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4e64f53fb124fa6b70e35bf156e2545, user_id=3f20604125354fd1b4a66fcc410da5a3, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:40.401267 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3065704-6dd1-4af9-96c6-0cb9aae791b8 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:40.428434 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 967/1932] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:40 2022] PUT /identity/v3/projects/d4e64f53fb124fa6b70e35bf156e2545/users/3f20604125354fd1b4a66fcc410da5a3/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:40.431071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3065704-6dd1-4af9-96c6-0cb9aae791b8 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:40.436496 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e3065704-6dd1-4af9-96c6-0cb9aae791b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:40.437408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674564e8-d927-43ac-911a-004d6dd77bba admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:40.533161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3065704-6dd1-4af9-96c6-0cb9aae791b8 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:40.535687 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c783c15-0018-4939-8c2a-269b59618d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:40.537183 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c783c15-0018-4939-8c2a-269b59618d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:40.537511 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c783c15-0018-4939-8c2a-269b59618d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:40.537923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c783c15-0018-4939-8c2a-269b59618d25 admin admin] PATH_INFO: `/v3/users/bb7595aed0a942539c906937b0543135` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:40.539789 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3794031-0408-4209-8476-f6520d6661e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:40.540445 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3794031-0408-4209-8476-f6520d6661e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:40.540733 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3794031-0408-4209-8476-f6520d6661e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:40.541014 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3794031-0408-4209-8476-f6520d6661e5 admin admin] PATH_INFO: `/v3/projects/a4e9e4be2bcc4ce59dd4ac80de3bdda3` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:40.541960 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c783c15-0018-4939-8c2a-269b59618d25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb7595aed0a942539c906937b0543135)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:40.542195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3794031-0408-4209-8476-f6520d6661e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4e9e4be2bcc4ce59dd4ac80de3bdda3)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:40.544191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3794031-0408-4209-8476-f6520d6661e5 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:40.545840 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c783c15-0018-4939-8c2a-269b59618d25 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:40.602221 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b3794031-0408-4209-8476-f6520d6661e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4e9e4be2bcc4ce59dd4ac80de3bdda3', 'request_id': 'req-b3794031-0408-4209-8476-f6520d6661e5'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:40.605091 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:40.612698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b3794031-0408-4209-8476-f6520d6661e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4e9e4be2bcc4ce59dd4ac80de3bdda3', 'request_id': 'req-b3794031-0408-4209-8476-f6520d6661e5'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:40.623864 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b3794031-0408-4209-8476-f6520d6661e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4e9e4be2bcc4ce59dd4ac80de3bdda3', 'request_id': 'req-b3794031-0408-4209-8476-f6520d6661e5'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:40.641377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b3794031-0408-4209-8476-f6520d6661e5 admin admin] The token cache is being invalidate because project a4e9e4be2bcc4ce59dd4ac80de3bdda3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:40.641830 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b3794031-0408-4209-8476-f6520d6661e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:40.642636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b3794031-0408-4209-8476-f6520d6661e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:40.644831 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 968/1933] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:40 2022] DELETE /identity/v3/projects/a4e9e4be2bcc4ce59dd4ac80de3bdda3 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:40.646410 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb0bff-5899-4487-b106-16c38003c1a9 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:40.646777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb0bff-5899-4487-b106-16c38003c1a9 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:40.647200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb0bff-5899-4487-b106-16c38003c1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:40.696574 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-62cb0bff-5899-4487-b106-16c38003c1a9 None None] MFA Rules not processed for user `3f20604125354fd1b4a66fcc410da5a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:40.698020 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-62cb0bff-5899-4487-b106-16c38003c1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:40.701949 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5c783c15-0018-4939-8c2a-269b59618d25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb7595aed0a942539c906937b0543135'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:40.703263 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5c783c15-0018-4939-8c2a-269b59618d25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb7595aed0a942539c906937b0543135'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:40.713584 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5c783c15-0018-4939-8c2a-269b59618d25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb7595aed0a942539c906937b0543135'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:40.714404 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5c783c15-0018-4939-8c2a-269b59618d25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb7595aed0a942539c906937b0543135'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:40.721944 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:40.726935 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5c783c15-0018-4939-8c2a-269b59618d25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb7595aed0a942539c906937b0543135'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:40.758203 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 966/1934] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:40 2022] DELETE /identity/v3/users/bb7595aed0a942539c906937b0543135 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:05:40.759248 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c783c15-0018-4939-8c2a-269b59618d25 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:40.760873 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5c783c15-0018-4939-8c2a-269b59618d25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:40.805677 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 969/1935] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:40.812135 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb0bff-5899-4487-b106-16c38003c1a9 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:40.813858 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-62cb0bff-5899-4487-b106-16c38003c1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:40.874280 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c783c15-0018-4939-8c2a-269b59618d25 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:40.887900 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dfd6e-1be0-4b76-955d-da3e9fd67f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:40.888573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dfd6e-1be0-4b76-955d-da3e9fd67f33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:40.891533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dfd6e-1be0-4b76-955d-da3e9fd67f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:40.891533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dfd6e-1be0-4b76-955d-da3e9fd67f33 admin admin] PATH_INFO: `/v3/users/097347b6e6a44160acc1d6a79798b4be` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:40.893844 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10dfd6e-1be0-4b76-955d-da3e9fd67f33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=097347b6e6a44160acc1d6a79798b4be)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:40.902089 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10dfd6e-1be0-4b76-955d-da3e9fd67f33 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:40.991110 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb0bff-5899-4487-b106-16c38003c1a9 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:41.003867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566146d5-dbb5-42a3-99c2-d1a2558888fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:41.004858 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566146d5-dbb5-42a3-99c2-d1a2558888fc admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:41.004858 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566146d5-dbb5-42a3-99c2-d1a2558888fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:41.005108 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566146d5-dbb5-42a3-99c2-d1a2558888fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:41.005768 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566146d5-dbb5-42a3-99c2-d1a2558888fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:41.013562 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566146d5-dbb5-42a3-99c2-d1a2558888fc admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:41.021274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b10dfd6e-1be0-4b76-955d-da3e9fd67f33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '097347b6e6a44160acc1d6a79798b4be'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:41.022115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b10dfd6e-1be0-4b76-955d-da3e9fd67f33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '097347b6e6a44160acc1d6a79798b4be'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:41.033727 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b10dfd6e-1be0-4b76-955d-da3e9fd67f33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '097347b6e6a44160acc1d6a79798b4be'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:41.034431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b10dfd6e-1be0-4b76-955d-da3e9fd67f33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '097347b6e6a44160acc1d6a79798b4be'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:41.052373 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b10dfd6e-1be0-4b76-955d-da3e9fd67f33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '097347b6e6a44160acc1d6a79798b4be'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:41.080757 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 970/1936] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:05:40 2022] POST /identity/v3/projects => generated 383 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:41.088725 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566146d5-dbb5-42a3-99c2-d1a2558888fc admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:41.090475 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-566146d5-dbb5-42a3-99c2-d1a2558888fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:41.091968 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 967/1937] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:40 2022] DELETE /identity/v3/users/097347b6e6a44160acc1d6a79798b4be => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:41.174493 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:41.189410 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566146d5-dbb5-42a3-99c2-d1a2558888fc admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:41.286735 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74584578-3826-4fe3-8564-2225c5d72ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:41.287446 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74584578-3826-4fe3-8564-2225c5d72ce3 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:41.287767 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74584578-3826-4fe3-8564-2225c5d72ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:41.288155 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74584578-3826-4fe3-8564-2225c5d72ce3 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:41.288790 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74584578-3826-4fe3-8564-2225c5d72ce3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:41.291002 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74584578-3826-4fe3-8564-2225c5d72ce3 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:41.319654 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 971/1938] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:05:41 2022] POST /identity/v3/users => generated 298 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:41.320931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dfd6e-1be0-4b76-955d-da3e9fd67f33 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:41.339540 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dfd6e-1be0-4b76-955d-da3e9fd67f33 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:41.346322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0ab0e-bcb6-48b6-b011-8789899eebc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:41.347297 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0ab0e-bcb6-48b6-b011-8789899eebc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:41.347800 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0ab0e-bcb6-48b6-b011-8789899eebc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:41.348296 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0ab0e-bcb6-48b6-b011-8789899eebc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:41.349222 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a0ab0e-bcb6-48b6-b011-8789899eebc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:41.351084 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a0ab0e-bcb6-48b6-b011-8789899eebc7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:41.382992 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 968/1939] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:41.384270 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74584578-3826-4fe3-8564-2225c5d72ce3 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:41.400223 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74584578-3826-4fe3-8564-2225c5d72ce3 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:41.410545 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d25083-74d7-4650-afb1-7f87be2a08be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:41.412393 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d25083-74d7-4650-afb1-7f87be2a08be admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:41.412978 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d25083-74d7-4650-afb1-7f87be2a08be admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:41.413499 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d25083-74d7-4650-afb1-7f87be2a08be admin admin] PATH_INFO: `/v3/projects/41e271f4396b43d59c2493b6fe83ebe8/users/f42bc8658ee74001a5a9179af9b117ab/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:41.441308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d25083-74d7-4650-afb1-7f87be2a08be admin admin] RBAC: Authorizing `identity:create_grant(project_id=41e271f4396b43d59c2493b6fe83ebe8, user_id=f42bc8658ee74001a5a9179af9b117ab, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:41.445545 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d25083-74d7-4650-afb1-7f87be2a08be admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:41.446618 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0ab0e-bcb6-48b6-b011-8789899eebc7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:41.448480 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c2a0ab0e-bcb6-48b6-b011-8789899eebc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:41.470245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 972/1940] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:41 2022] PUT /identity/v3/projects/41e271f4396b43d59c2493b6fe83ebe8/users/f42bc8658ee74001a5a9179af9b117ab/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:41.474803 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d25083-74d7-4650-afb1-7f87be2a08be admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:41.475011 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0ab0e-bcb6-48b6-b011-8789899eebc7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:41.540501 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:41.565028 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03331894-0369-47e8-af43-12ee4eb00de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:41.565694 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03331894-0369-47e8-af43-12ee4eb00de2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:41.566105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03331894-0369-47e8-af43-12ee4eb00de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:41.566724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03331894-0369-47e8-af43-12ee4eb00de2 admin admin] PATH_INFO: `/v3/projects/3cd778ba2cd24d54ba42007c51c30cd8` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:41.568426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03331894-0369-47e8-af43-12ee4eb00de2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cd778ba2cd24d54ba42007c51c30cd8)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:41.570798 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03331894-0369-47e8-af43-12ee4eb00de2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:41.575909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d25083-74d7-4650-afb1-7f87be2a08be admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:41.585168 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9200e84-f4a2-4a43-9cf0-5b50b5fbd930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:41.585813 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9200e84-f4a2-4a43-9cf0-5b50b5fbd930 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:41.586100 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9200e84-f4a2-4a43-9cf0-5b50b5fbd930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:41.586375 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9200e84-f4a2-4a43-9cf0-5b50b5fbd930 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:41.587040 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9200e84-f4a2-4a43-9cf0-5b50b5fbd930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:41.588807 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9200e84-f4a2-4a43-9cf0-5b50b5fbd930 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:41.603423 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 973/1941] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:41.606610 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9200e84-f4a2-4a43-9cf0-5b50b5fbd930 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:41.621117 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9200e84-f4a2-4a43-9cf0-5b50b5fbd930 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:41.627357 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127a6758-200e-490c-9604-dc031714ea02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:41.628297 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a6758-200e-490c-9604-dc031714ea02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:41.628704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a6758-200e-490c-9604-dc031714ea02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:41.632120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a6758-200e-490c-9604-dc031714ea02 admin admin] PATH_INFO: `/v3/projects/41e271f4396b43d59c2493b6fe83ebe8/users/f42bc8658ee74001a5a9179af9b117ab/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:41.634574 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127a6758-200e-490c-9604-dc031714ea02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41e271f4396b43d59c2493b6fe83ebe8, user_id=f42bc8658ee74001a5a9179af9b117ab, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:41.637754 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127a6758-200e-490c-9604-dc031714ea02 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:41.650728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-03331894-0369-47e8-af43-12ee4eb00de2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cd778ba2cd24d54ba42007c51c30cd8', 'request_id': 'req-03331894-0369-47e8-af43-12ee4eb00de2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:41.660649 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 974/1942] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:41 2022] PUT /identity/v3/projects/41e271f4396b43d59c2493b6fe83ebe8/users/f42bc8658ee74001a5a9179af9b117ab/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:05:41.660799 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-03331894-0369-47e8-af43-12ee4eb00de2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cd778ba2cd24d54ba42007c51c30cd8', 'request_id': 'req-03331894-0369-47e8-af43-12ee4eb00de2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:41.661807 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127a6758-200e-490c-9604-dc031714ea02 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:41.664730 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-127a6758-200e-490c-9604-dc031714ea02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:41.672779 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-03331894-0369-47e8-af43-12ee4eb00de2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cd778ba2cd24d54ba42007c51c30cd8', 'request_id': 'req-03331894-0369-47e8-af43-12ee4eb00de2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:41.688919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-03331894-0369-47e8-af43-12ee4eb00de2 admin admin] The token cache is being invalidate because project 3cd778ba2cd24d54ba42007c51c30cd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:41.689208 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-03331894-0369-47e8-af43-12ee4eb00de2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:41.689875 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-03331894-0369-47e8-af43-12ee4eb00de2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:41.691689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 969/1943] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:41 2022] DELETE /identity/v3/projects/3cd778ba2cd24d54ba42007c51c30cd8 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:41.694506 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6612623-d243-407f-8f0c-d29c8efc2148 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:41.694506 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6612623-d243-407f-8f0c-d29c8efc2148 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:41.694717 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6612623-d243-407f-8f0c-d29c8efc2148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:41.744986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:41.763805 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f6612623-d243-407f-8f0c-d29c8efc2148 None None] MFA Rules not processed for user `f42bc8658ee74001a5a9179af9b117ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:41.765613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f6612623-d243-407f-8f0c-d29c8efc2148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:41.767608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127a6758-200e-490c-9604-dc031714ea02 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:41.774819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f780e7-3bee-4f03-8e45-3a2ef88ca67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:41.775573 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f780e7-3bee-4f03-8e45-3a2ef88ca67e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:41.776081 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f780e7-3bee-4f03-8e45-3a2ef88ca67e admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:41.776362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f780e7-3bee-4f03-8e45-3a2ef88ca67e admin admin] PATH_INFO: `/v3/projects/2f2f85f939544eeead876788ac3b0663` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:41.778067 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f780e7-3bee-4f03-8e45-3a2ef88ca67e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f2f85f939544eeead876788ac3b0663)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:41.781023 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f780e7-3bee-4f03-8e45-3a2ef88ca67e admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:41.839377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a6f780e7-3bee-4f03-8e45-3a2ef88ca67e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f2f85f939544eeead876788ac3b0663', 'request_id': 'req-a6f780e7-3bee-4f03-8e45-3a2ef88ca67e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:41.848759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a6f780e7-3bee-4f03-8e45-3a2ef88ca67e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f2f85f939544eeead876788ac3b0663', 'request_id': 'req-a6f780e7-3bee-4f03-8e45-3a2ef88ca67e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:41.850067 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 970/1944] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:41 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:41.850966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6612623-d243-407f-8f0c-d29c8efc2148 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:41.852357 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f6612623-d243-407f-8f0c-d29c8efc2148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:41.860028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a6f780e7-3bee-4f03-8e45-3a2ef88ca67e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f2f85f939544eeead876788ac3b0663', 'request_id': 'req-a6f780e7-3bee-4f03-8e45-3a2ef88ca67e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:41.867084 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6612623-d243-407f-8f0c-d29c8efc2148 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:41.873831 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a6f780e7-3bee-4f03-8e45-3a2ef88ca67e admin admin] The token cache is being invalidate because project 2f2f85f939544eeead876788ac3b0663 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:41.874123 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a6f780e7-3bee-4f03-8e45-3a2ef88ca67e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:41.874840 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a6f780e7-3bee-4f03-8e45-3a2ef88ca67e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:41.876771 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 975/1945] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:41 2022] DELETE /identity/v3/projects/2f2f85f939544eeead876788ac3b0663 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:41.877672 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f780e7-3bee-4f03-8e45-3a2ef88ca67e admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:41.878994 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a6f780e7-3bee-4f03-8e45-3a2ef88ca67e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:41.939792 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643aa3d2-e0e9-4efa-ba6e-bfe9fe85ee5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:41.940747 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643aa3d2-e0e9-4efa-ba6e-bfe9fe85ee5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:41.941028 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643aa3d2-e0e9-4efa-ba6e-bfe9fe85ee5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:41.941302 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643aa3d2-e0e9-4efa-ba6e-bfe9fe85ee5f admin admin] PATH_INFO: `/v3/users/0da396e6cebc4abdab4f96b4f09d6525` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:41.943002 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643aa3d2-e0e9-4efa-ba6e-bfe9fe85ee5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0da396e6cebc4abdab4f96b4f09d6525)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:41.944205 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f780e7-3bee-4f03-8e45-3a2ef88ca67e admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:41.944967 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643aa3d2-e0e9-4efa-ba6e-bfe9fe85ee5f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:41.947304 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:41.948509 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4106c7b6-ff48-4a08-9b05-9aa9fea05775 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:41.949132 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106c7b6-ff48-4a08-9b05-9aa9fea05775 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:41.949390 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106c7b6-ff48-4a08-9b05-9aa9fea05775 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:41.949687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106c7b6-ff48-4a08-9b05-9aa9fea05775 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:41.952084 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4106c7b6-ff48-4a08-9b05-9aa9fea05775 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:41.998691 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-643aa3d2-e0e9-4efa-ba6e-bfe9fe85ee5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0da396e6cebc4abdab4f96b4f09d6525'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:41.999411 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-643aa3d2-e0e9-4efa-ba6e-bfe9fe85ee5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0da396e6cebc4abdab4f96b4f09d6525'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:42.009066 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-643aa3d2-e0e9-4efa-ba6e-bfe9fe85ee5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0da396e6cebc4abdab4f96b4f09d6525'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:42.009811 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-643aa3d2-e0e9-4efa-ba6e-bfe9fe85ee5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0da396e6cebc4abdab4f96b4f09d6525'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:42.018651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-643aa3d2-e0e9-4efa-ba6e-bfe9fe85ee5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0da396e6cebc4abdab4f96b4f09d6525'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:42.019106 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4106c7b6-ff48-4a08-9b05-9aa9fea05775 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:42.021397 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4106c7b6-ff48-4a08-9b05-9aa9fea05775 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:42.032716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 976/1946] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:41 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:42.033855 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5a97e-4849-4c22-a1c8-31e8f4abe12a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:42.034144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5a97e-4849-4c22-a1c8-31e8f4abe12a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:42.034421 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5a97e-4849-4c22-a1c8-31e8f4abe12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:42.041069 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 971/1947] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:41 2022] DELETE /identity/v3/users/0da396e6cebc4abdab4f96b4f09d6525 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:42.055740 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-05c5a97e-4849-4c22-a1c8-31e8f4abe12a None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:42.057276 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-05c5a97e-4849-4c22-a1c8-31e8f4abe12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:42.106213 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:42.194963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 977/1948] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:42.195059 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68108f7-b7b0-4d4e-84ce-c5f97f2ddfa6 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:42.195059 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68108f7-b7b0-4d4e-84ce-c5f97f2ddfa6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:42.195059 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68108f7-b7b0-4d4e-84ce-c5f97f2ddfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:42.204363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5a97e-4849-4c22-a1c8-31e8f4abe12a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:42.207135 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-05c5a97e-4849-4c22-a1c8-31e8f4abe12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:42.222656 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a68108f7-b7b0-4d4e-84ce-c5f97f2ddfa6 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:42.225211 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a68108f7-b7b0-4d4e-84ce-c5f97f2ddfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:42.237701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 972/1949] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:05:42.242176 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68108f7-b7b0-4d4e-84ce-c5f97f2ddfa6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:42.258018 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68108f7-b7b0-4d4e-84ce-c5f97f2ddfa6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:42.264100 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2685a57-91bf-4d8f-b5a3-a3c8e90e6bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:42.264985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2685a57-91bf-4d8f-b5a3-a3c8e90e6bae admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:42.265260 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2685a57-91bf-4d8f-b5a3-a3c8e90e6bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:42.265526 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2685a57-91bf-4d8f-b5a3-a3c8e90e6bae admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:42.266287 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2685a57-91bf-4d8f-b5a3-a3c8e90e6bae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:05:42.266599 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2685a57-91bf-4d8f-b5a3-a3c8e90e6bae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:42.268864 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2685a57-91bf-4d8f-b5a3-a3c8e90e6bae admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:42.283310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5a97e-4849-4c22-a1c8-31e8f4abe12a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:42.289823 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67341aa-ca9b-4452-b163-617a29319a34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:42.291144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67341aa-ca9b-4452-b163-617a29319a34 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:42.291674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67341aa-ca9b-4452-b163-617a29319a34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:42.292162 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67341aa-ca9b-4452-b163-617a29319a34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:42.297132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 973/1950] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:05:42.297329 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e67341aa-ca9b-4452-b163-617a29319a34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:42.298203 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2685a57-91bf-4d8f-b5a3-a3c8e90e6bae admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:42.313440 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2685a57-91bf-4d8f-b5a3-a3c8e90e6bae admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:42.320004 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8a4bf-313c-45ac-ba79-5d74138850e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:42.320624 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8a4bf-313c-45ac-ba79-5d74138850e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:42.323500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8a4bf-313c-45ac-ba79-5d74138850e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:42.323926 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8a4bf-313c-45ac-ba79-5d74138850e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:42.324644 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8a4bf-313c-45ac-ba79-5d74138850e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:42.326828 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8a4bf-313c-45ac-ba79-5d74138850e5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:42.381922 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 974/1951] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:05:42 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:05:42.385833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8a4bf-313c-45ac-ba79-5d74138850e5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:42.412778 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67341aa-ca9b-4452-b163-617a29319a34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:42.414248 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67341aa-ca9b-4452-b163-617a29319a34 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:42.441146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 978/1952] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:05:42.442020 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67341aa-ca9b-4452-b163-617a29319a34 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:42.444440 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e67341aa-ca9b-4452-b163-617a29319a34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:42.481090 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67341aa-ca9b-4452-b163-617a29319a34 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:42.515741 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8a4bf-313c-45ac-ba79-5d74138850e5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:42.523503 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5621b-e80e-4993-a3b3-496f3ad64204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:42.524537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5621b-e80e-4993-a3b3-496f3ad64204 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:42.525294 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5621b-e80e-4993-a3b3-496f3ad64204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:42.525952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5621b-e80e-4993-a3b3-496f3ad64204 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:42.527147 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca5621b-e80e-4993-a3b3-496f3ad64204 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:42.530608 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca5621b-e80e-4993-a3b3-496f3ad64204 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:42.563644 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 975/1953] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:05:42 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:42.569473 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5621b-e80e-4993-a3b3-496f3ad64204 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:42.586016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5621b-e80e-4993-a3b3-496f3ad64204 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:42.590681 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c264e-c4eb-4eaa-9f04-86375c3ace48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:42.591336 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c264e-c4eb-4eaa-9f04-86375c3ace48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:42.592016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c264e-c4eb-4eaa-9f04-86375c3ace48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:42.592199 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c264e-c4eb-4eaa-9f04-86375c3ace48 admin admin] PATH_INFO: `/v3/projects/81c7ee8953e54b0fbfb16e612573f832` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:42.594570 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c264e-c4eb-4eaa-9f04-86375c3ace48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81c7ee8953e54b0fbfb16e612573f832)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:42.596553 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c264e-c4eb-4eaa-9f04-86375c3ace48 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:42.604794 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d75fbb-8dc6-4cd8-ae7e-ea614edcd265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:42.606489 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d75fbb-8dc6-4cd8-ae7e-ea614edcd265 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:42.607001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d75fbb-8dc6-4cd8-ae7e-ea614edcd265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:42.607479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d75fbb-8dc6-4cd8-ae7e-ea614edcd265 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:42.608390 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d75fbb-8dc6-4cd8-ae7e-ea614edcd265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:42.610758 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d75fbb-8dc6-4cd8-ae7e-ea614edcd265 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:42.623090 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 976/1954] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:42.627121 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d75fbb-8dc6-4cd8-ae7e-ea614edcd265 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:42.637883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d75fbb-8dc6-4cd8-ae7e-ea614edcd265 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:42.644773 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add3adc-8d69-40c5-bf52-339a775e452a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:42.645545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add3adc-8d69-40c5-bf52-339a775e452a admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:42.646118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add3adc-8d69-40c5-bf52-339a775e452a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:42.646493 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add3adc-8d69-40c5-bf52-339a775e452a admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:42.647250 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add3adc-8d69-40c5-bf52-339a775e452a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:42.649067 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add3adc-8d69-40c5-bf52-339a775e452a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:42.662367 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 977/1955] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:05:42.667501 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add3adc-8d69-40c5-bf52-339a775e452a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:42.668680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:42.693649 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add3adc-8d69-40c5-bf52-339a775e452a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:42.696830 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-632c264e-c4eb-4eaa-9f04-86375c3ace48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81c7ee8953e54b0fbfb16e612573f832', 'request_id': 'req-632c264e-c4eb-4eaa-9f04-86375c3ace48'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:42.699404 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:42.708462 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-632c264e-c4eb-4eaa-9f04-86375c3ace48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81c7ee8953e54b0fbfb16e612573f832', 'request_id': 'req-632c264e-c4eb-4eaa-9f04-86375c3ace48'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:42.718225 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-632c264e-c4eb-4eaa-9f04-86375c3ace48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81c7ee8953e54b0fbfb16e612573f832', 'request_id': 'req-632c264e-c4eb-4eaa-9f04-86375c3ace48'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:42.735543 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-632c264e-c4eb-4eaa-9f04-86375c3ace48 admin admin] The token cache is being invalidate because project 81c7ee8953e54b0fbfb16e612573f832 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:42.735873 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-632c264e-c4eb-4eaa-9f04-86375c3ace48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:42.737281 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-632c264e-c4eb-4eaa-9f04-86375c3ace48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:42.739969 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 979/1956] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:42 2022] DELETE /identity/v3/projects/81c7ee8953e54b0fbfb16e612573f832 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:42.744669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42bfac1-6a20-4748-8c4d-6a494fa26ec8 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:42.745137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42bfac1-6a20-4748-8c4d-6a494fa26ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:42.745446 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42bfac1-6a20-4748-8c4d-6a494fa26ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:42.774263 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ff0ca-1c5d-4a67-b2c7-ce7b19452763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:42.774952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ff0ca-1c5d-4a67-b2c7-ce7b19452763 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:42.775239 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ff0ca-1c5d-4a67-b2c7-ce7b19452763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:42.775532 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ff0ca-1c5d-4a67-b2c7-ce7b19452763 admin admin] PATH_INFO: `/v3/projects/a1a99d8966c14722b49f35d014e203c5/users/c7bc8a31251f4112b078811fcd211a5e/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:42.776213 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d42bfac1-6a20-4748-8c4d-6a494fa26ec8 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:42.777646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d42bfac1-6a20-4748-8c4d-6a494fa26ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:42.792657 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ff0ca-1c5d-4a67-b2c7-ce7b19452763 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a99d8966c14722b49f35d014e203c5, user_id=c7bc8a31251f4112b078811fcd211a5e, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:42.794425 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 980/1957] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:42.795522 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ff0ca-1c5d-4a67-b2c7-ce7b19452763 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:42.809698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e66901-7179-4181-b7fd-e260f74a3158 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:42.810146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e66901-7179-4181-b7fd-e260f74a3158 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:42.810646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e66901-7179-4181-b7fd-e260f74a3158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:42.827947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 978/1958] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:42 2022] PUT /identity/v3/projects/a1a99d8966c14722b49f35d014e203c5/users/c7bc8a31251f4112b078811fcd211a5e/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:42.829782 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951af52d-6e86-48b5-aee4-dc5e023e71f4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:42.830103 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951af52d-6e86-48b5-aee4-dc5e023e71f4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:42.830382 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951af52d-6e86-48b5-aee4-dc5e023e71f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:42.834036 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d5e66901-7179-4181-b7fd-e260f74a3158 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:42.835577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d5e66901-7179-4181-b7fd-e260f74a3158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:42.868383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-951af52d-6e86-48b5-aee4-dc5e023e71f4 None None] MFA Rules not processed for user `c7bc8a31251f4112b078811fcd211a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:42.869711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-951af52d-6e86-48b5-aee4-dc5e023e71f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:42.970808 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 981/1959] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:42.979238 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e66901-7179-4181-b7fd-e260f74a3158 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:43.001839 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e66901-7179-4181-b7fd-e260f74a3158 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:43.009242 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948324c-16c5-45ce-a016-5ca6009686ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:43.009873 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948324c-16c5-45ce-a016-5ca6009686ae admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:43.010162 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948324c-16c5-45ce-a016-5ca6009686ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:43.010639 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948324c-16c5-45ce-a016-5ca6009686ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:43.011340 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948324c-16c5-45ce-a016-5ca6009686ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:05:43.011673 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948324c-16c5-45ce-a016-5ca6009686ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:43.015950 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948324c-16c5-45ce-a016-5ca6009686ae admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:43.029520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 979/1960] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:05:43.032027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 982/1961] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:05:43.035523 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951af52d-6e86-48b5-aee4-dc5e023e71f4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:43.036407 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948324c-16c5-45ce-a016-5ca6009686ae admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:43.037768 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-951af52d-6e86-48b5-aee4-dc5e023e71f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:43.046691 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948324c-16c5-45ce-a016-5ca6009686ae admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:43.052879 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120e256-0cef-4fe6-9a30-ba391d3eec67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:43.053823 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120e256-0cef-4fe6-9a30-ba391d3eec67 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:43.054107 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120e256-0cef-4fe6-9a30-ba391d3eec67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:43.054378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120e256-0cef-4fe6-9a30-ba391d3eec67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:43.055578 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951af52d-6e86-48b5-aee4-dc5e023e71f4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:43.055725 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2120e256-0cef-4fe6-9a30-ba391d3eec67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:43.057518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2120e256-0cef-4fe6-9a30-ba391d3eec67 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:43.061457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6e07a-1ed2-4ed1-b393-6ae5b9a4deb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:43.062470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6e07a-1ed2-4ed1-b393-6ae5b9a4deb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:43.063078 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6e07a-1ed2-4ed1-b393-6ae5b9a4deb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:43.063501 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6e07a-1ed2-4ed1-b393-6ae5b9a4deb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:43.064335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6e07a-1ed2-4ed1-b393-6ae5b9a4deb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:43.066388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6e07a-1ed2-4ed1-b393-6ae5b9a4deb2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:43.100855 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 983/1962] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:05:43 2022] POST /identity/v3/projects => generated 367 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:05:43.103101 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 980/1963] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:05:43 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:43.106952 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120e256-0cef-4fe6-9a30-ba391d3eec67 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:43.107456 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6e07a-1ed2-4ed1-b393-6ae5b9a4deb2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:43.172882 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:43.177313 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6e07a-1ed2-4ed1-b393-6ae5b9a4deb2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:43.180641 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120e256-0cef-4fe6-9a30-ba391d3eec67 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:43.183458 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb76352-87a7-4110-b5ea-a30d63d08d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:43.184108 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb76352-87a7-4110-b5ea-a30d63d08d2c admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:43.184452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb76352-87a7-4110-b5ea-a30d63d08d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:43.184710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb76352-87a7-4110-b5ea-a30d63d08d2c admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:43.185349 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb76352-87a7-4110-b5ea-a30d63d08d2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:43.187015 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993465d-2825-4e11-af25-241f1de82d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:43.187424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb76352-87a7-4110-b5ea-a30d63d08d2c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:43.187681 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993465d-2825-4e11-af25-241f1de82d7f admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:43.187965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993465d-2825-4e11-af25-241f1de82d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:43.188236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993465d-2825-4e11-af25-241f1de82d7f admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:43.188879 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993465d-2825-4e11-af25-241f1de82d7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:43.190780 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993465d-2825-4e11-af25-241f1de82d7f admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:43.210146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 981/1964] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:05:43 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:05:43.214046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb76352-87a7-4110-b5ea-a30d63d08d2c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:43.215586 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 984/1965] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:05:43 2022] POST /identity/v3/users => generated 284 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:05:43.219341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993465d-2825-4e11-af25-241f1de82d7f admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:43.224280 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb76352-87a7-4110-b5ea-a30d63d08d2c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:43.225691 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:43.229296 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5b64f-6efd-49d4-840f-c2e012672ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:43.229687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993465d-2825-4e11-af25-241f1de82d7f admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:43.229901 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5b64f-6efd-49d4-840f-c2e012672ac0 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:43.230183 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5b64f-6efd-49d4-840f-c2e012672ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:43.230522 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5b64f-6efd-49d4-840f-c2e012672ac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:43.231173 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5b64f-6efd-49d4-840f-c2e012672ac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:43.232843 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5b64f-6efd-49d4-840f-c2e012672ac0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:43.235485 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18c8c0-dd5a-4ba7-96f9-d82654586ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:43.236859 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18c8c0-dd5a-4ba7-96f9-d82654586ede admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:43.237300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18c8c0-dd5a-4ba7-96f9-d82654586ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:43.237635 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18c8c0-dd5a-4ba7-96f9-d82654586ede admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:43.238491 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18c8c0-dd5a-4ba7-96f9-d82654586ede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:43.240179 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18c8c0-dd5a-4ba7-96f9-d82654586ede admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:43.244313 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 982/1966] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:05:43.248359 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5b64f-6efd-49d4-840f-c2e012672ac0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:43.253363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 985/1967] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:05:43.255037 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18c8c0-dd5a-4ba7-96f9-d82654586ede admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:43.261890 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5b64f-6efd-49d4-840f-c2e012672ac0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:43.265661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18c8c0-dd5a-4ba7-96f9-d82654586ede admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:43.267287 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6d097-c398-4a0b-8593-410ba3c424e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:43.267957 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6d097-c398-4a0b-8593-410ba3c424e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:43.268265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6d097-c398-4a0b-8593-410ba3c424e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:43.268519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6d097-c398-4a0b-8593-410ba3c424e8 admin admin] PATH_INFO: `/v3/projects/83f26fe886fe41f8bd41ef54a54b8675/users/3ad4f2ea64df4c71994fd9f69811cc55/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:43.270499 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919dde46-9724-4769-9941-01c22ec7b61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:43.271266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919dde46-9724-4769-9941-01c22ec7b61a admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:43.271821 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919dde46-9724-4769-9941-01c22ec7b61a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:43.272996 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919dde46-9724-4769-9941-01c22ec7b61a admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:43.272996 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919dde46-9724-4769-9941-01c22ec7b61a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:43.274693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919dde46-9724-4769-9941-01c22ec7b61a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:43.287593 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 986/1968] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:05:43.291925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919dde46-9724-4769-9941-01c22ec7b61a admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:43.292293 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6d097-c398-4a0b-8593-410ba3c424e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83f26fe886fe41f8bd41ef54a54b8675, user_id=3ad4f2ea64df4c71994fd9f69811cc55, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:43.295028 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6d097-c398-4a0b-8593-410ba3c424e8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:43.304146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919dde46-9724-4769-9941-01c22ec7b61a admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:43.309883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 983/1969] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:43 2022] PUT /identity/v3/projects/83f26fe886fe41f8bd41ef54a54b8675/users/3ad4f2ea64df4c71994fd9f69811cc55/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:05:43.313699 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6d097-c398-4a0b-8593-410ba3c424e8 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:43.368744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c5ae5-8015-40cd-bd1e-288aceec86a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:43.369350 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c5ae5-8015-40cd-bd1e-288aceec86a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:43.369629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c5ae5-8015-40cd-bd1e-288aceec86a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:43.369903 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c5ae5-8015-40cd-bd1e-288aceec86a2 admin admin] PATH_INFO: `/v3/projects/38e0fb44599648b99e9d59701bca4a90/users/be12ec8762ec414892642843701821ce/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:43.382663 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6d097-c398-4a0b-8593-410ba3c424e8 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:43.386648 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c5ae5-8015-40cd-bd1e-288aceec86a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38e0fb44599648b99e9d59701bca4a90, user_id=be12ec8762ec414892642843701821ce, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:43.388429 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e109a-c520-40c8-af3d-8239c34710b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:43.389052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e109a-c520-40c8-af3d-8239c34710b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:43.389593 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e109a-c520-40c8-af3d-8239c34710b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:43.389693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c5ae5-8015-40cd-bd1e-288aceec86a2 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:43.389870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e109a-c520-40c8-af3d-8239c34710b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:43.390721 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e109a-c520-40c8-af3d-8239c34710b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:43.392156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e109a-c520-40c8-af3d-8239c34710b4 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:43.408110 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 987/1970] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:43 2022] PUT /identity/v3/projects/38e0fb44599648b99e9d59701bca4a90/users/be12ec8762ec414892642843701821ce/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:05:43.408525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 984/1971] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:05:43.409259 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c5ae5-8015-40cd-bd1e-288aceec86a2 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:43.416179 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950048f2-dcea-4a43-bee2-6b63c3b9d1c4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:43.416662 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950048f2-dcea-4a43-bee2-6b63c3b9d1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:43.417122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950048f2-dcea-4a43-bee2-6b63c3b9d1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:43.454585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-950048f2-dcea-4a43-bee2-6b63c3b9d1c4 None None] MFA Rules not processed for user `be12ec8762ec414892642843701821ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:43.455970 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-950048f2-dcea-4a43-bee2-6b63c3b9d1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:43.484992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c5ae5-8015-40cd-bd1e-288aceec86a2 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:43.491543 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68b251-80aa-4441-86e1-27699e5d4897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:43.492241 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68b251-80aa-4441-86e1-27699e5d4897 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:43.492524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68b251-80aa-4441-86e1-27699e5d4897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:43.492799 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68b251-80aa-4441-86e1-27699e5d4897 admin admin] PATH_INFO: `/v3/projects/83f26fe886fe41f8bd41ef54a54b8675/users/3ad4f2ea64df4c71994fd9f69811cc55/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:43.494738 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68b251-80aa-4441-86e1-27699e5d4897 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83f26fe886fe41f8bd41ef54a54b8675, user_id=3ad4f2ea64df4c71994fd9f69811cc55, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:43.497452 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68b251-80aa-4441-86e1-27699e5d4897 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:43.508538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 988/1972] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:43 2022] PUT /identity/v3/projects/83f26fe886fe41f8bd41ef54a54b8675/users/3ad4f2ea64df4c71994fd9f69811cc55/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:43.518947 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159170d-4537-40d9-b1f7-c179bf832ace None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:43.519238 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159170d-4537-40d9-b1f7-c179bf832ace None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:43.519519 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159170d-4537-40d9-b1f7-c179bf832ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:43.529025 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 985/1973] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:43.541264 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950048f2-dcea-4a43-bee2-6b63c3b9d1c4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:43.542648 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-950048f2-dcea-4a43-bee2-6b63c3b9d1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:43.547583 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-b159170d-4537-40d9-b1f7-c179bf832ace None None] MFA Rules not processed for user `3ad4f2ea64df4c71994fd9f69811cc55`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:43.548764 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b159170d-4537-40d9-b1f7-c179bf832ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:43.587103 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:43.597571 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:43.624395 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 989/1974] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:05:43.627800 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950048f2-dcea-4a43-bee2-6b63c3b9d1c4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:43.639117 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159170d-4537-40d9-b1f7-c179bf832ace None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:43.640931 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b159170d-4537-40d9-b1f7-c179bf832ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:43.683344 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d606f1-70fa-4dc7-9445-9fdb139593ec service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89ff357209194276bc079d0c05898747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:43.684009 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d606f1-70fa-4dc7-9445-9fdb139593ec service glance] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:43.684304 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d606f1-70fa-4dc7-9445-9fdb139593ec service glance] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:43.684579 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d606f1-70fa-4dc7-9445-9fdb139593ec service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:43.714358 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159170d-4537-40d9-b1f7-c179bf832ace None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:43.718360 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369726a-3947-4051-b43a-9be9866732ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:43.719008 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369726a-3947-4051-b43a-9be9866732ca service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:43.719287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369726a-3947-4051-b43a-9be9866732ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:43.719592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369726a-3947-4051-b43a-9be9866732ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:43.732360 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369726a-3947-4051-b43a-9be9866732ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:43.734132 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369726a-3947-4051-b43a-9be9866732ca service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:43.745430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 990/1975] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:05:43.750843 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d606f1-70fa-4dc7-9445-9fdb139593ec service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:43.752353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d606f1-70fa-4dc7-9445-9fdb139593ec service glance] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:43.762270 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 986/1976] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:44.679873 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369726a-3947-4051-b43a-9be9866732ca service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:44.770918 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369726a-3947-4051-b43a-9be9866732ca service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:44.776734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcdb518-8ce4-4902-9677-545e8ec758cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:44.777455 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdb518-8ce4-4902-9677-545e8ec758cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:44.779161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdb518-8ce4-4902-9677-545e8ec758cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:44.779512 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdb518-8ce4-4902-9677-545e8ec758cc admin admin] PATH_INFO: `/v3/users/c7bc8a31251f4112b078811fcd211a5e` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:44.780784 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcdb518-8ce4-4902-9677-545e8ec758cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7bc8a31251f4112b078811fcd211a5e)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:44.782811 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcdb518-8ce4-4902-9677-545e8ec758cc admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:44.841868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4bcdb518-8ce4-4902-9677-545e8ec758cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7bc8a31251f4112b078811fcd211a5e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:44.842801 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4bcdb518-8ce4-4902-9677-545e8ec758cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7bc8a31251f4112b078811fcd211a5e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:44.856828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4bcdb518-8ce4-4902-9677-545e8ec758cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7bc8a31251f4112b078811fcd211a5e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:44.873652 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4bcdb518-8ce4-4902-9677-545e8ec758cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7bc8a31251f4112b078811fcd211a5e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:44.875281 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4bcdb518-8ce4-4902-9677-545e8ec758cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7bc8a31251f4112b078811fcd211a5e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:44.945034 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 991/1977] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:44 2022] DELETE /identity/v3/users/c7bc8a31251f4112b078811fcd211a5e => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:44.949950 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef3c5b-89b1-4867-8a49-d47ada12eca6 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:44.950460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef3c5b-89b1-4867-8a49-d47ada12eca6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:44.950590 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef3c5b-89b1-4867-8a49-d47ada12eca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:44.982042 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-29ef3c5b-89b1-4867-8a49-d47ada12eca6 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:44.984692 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-29ef3c5b-89b1-4867-8a49-d47ada12eca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:45.134577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 987/1978] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:45.156811 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcdb518-8ce4-4902-9677-545e8ec758cc admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:45.160933 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4bcdb518-8ce4-4902-9677-545e8ec758cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:45.180693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:45.290825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcdb518-8ce4-4902-9677-545e8ec758cc admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:45.306903 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7426f5-4502-4418-b579-caf787a1870a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:45.306903 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7426f5-4502-4418-b579-caf787a1870a service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:45.306903 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7426f5-4502-4418-b579-caf787a1870a service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:45.306903 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7426f5-4502-4418-b579-caf787a1870a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:45.327703 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7426f5-4502-4418-b579-caf787a1870a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:45.332683 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7426f5-4502-4418-b579-caf787a1870a service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:45.347460 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 992/1979] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:45.608249 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef3c5b-89b1-4867-8a49-d47ada12eca6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:45.609752 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-29ef3c5b-89b1-4867-8a49-d47ada12eca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:45.626779 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef3c5b-89b1-4867-8a49-d47ada12eca6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:45.636423 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c05e85-34c5-445d-9076-7addc7ffbf24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:45.637236 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c05e85-34c5-445d-9076-7addc7ffbf24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:45.637529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c05e85-34c5-445d-9076-7addc7ffbf24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:45.637800 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c05e85-34c5-445d-9076-7addc7ffbf24 admin admin] PATH_INFO: `/v3/users/3f20604125354fd1b4a66fcc410da5a3` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:45.639632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c05e85-34c5-445d-9076-7addc7ffbf24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f20604125354fd1b4a66fcc410da5a3)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:45.641833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c05e85-34c5-445d-9076-7addc7ffbf24 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:45.676055 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:45.719092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-52c05e85-34c5-445d-9076-7addc7ffbf24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f20604125354fd1b4a66fcc410da5a3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:45.719819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-52c05e85-34c5-445d-9076-7addc7ffbf24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f20604125354fd1b4a66fcc410da5a3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:45.730209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-52c05e85-34c5-445d-9076-7addc7ffbf24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f20604125354fd1b4a66fcc410da5a3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:45.731728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-52c05e85-34c5-445d-9076-7addc7ffbf24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f20604125354fd1b4a66fcc410da5a3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:45.742807 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-52c05e85-34c5-445d-9076-7addc7ffbf24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f20604125354fd1b4a66fcc410da5a3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:45.768023 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 988/1980] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:45 2022] DELETE /identity/v3/users/3f20604125354fd1b4a66fcc410da5a3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:45.775595 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdec887-3c89-4c0b-909f-5837bba18e0e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:45.776166 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdec887-3c89-4c0b-909f-5837bba18e0e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:45.776266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdec887-3c89-4c0b-909f-5837bba18e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:45.795219 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c05e85-34c5-445d-9076-7addc7ffbf24 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:45.797233 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-52c05e85-34c5-445d-9076-7addc7ffbf24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:45.798202 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-ecdec887-3c89-4c0b-909f-5837bba18e0e None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:45.799430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ecdec887-3c89-4c0b-909f-5837bba18e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:45.878159 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 993/1981] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:45.886165 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c05e85-34c5-445d-9076-7addc7ffbf24 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:45.888118 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdec887-3c89-4c0b-909f-5837bba18e0e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:45.889791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ecdec887-3c89-4c0b-909f-5837bba18e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:45.891658 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66e0f8-77c3-4a44-b876-b167a9d4234e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:45.892303 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66e0f8-77c3-4a44-b876-b167a9d4234e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:45.892587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66e0f8-77c3-4a44-b876-b167a9d4234e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:45.892867 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66e0f8-77c3-4a44-b876-b167a9d4234e admin admin] PATH_INFO: `/v3/projects/a1a99d8966c14722b49f35d014e203c5` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:45.894002 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc66e0f8-77c3-4a44-b876-b167a9d4234e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1a99d8966c14722b49f35d014e203c5)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:45.896715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc66e0f8-77c3-4a44-b876-b167a9d4234e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:45.948430 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-cc66e0f8-77c3-4a44-b876-b167a9d4234e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1a99d8966c14722b49f35d014e203c5', 'request_id': 'req-cc66e0f8-77c3-4a44-b876-b167a9d4234e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:45.952538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdec887-3c89-4c0b-909f-5837bba18e0e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:45.956201 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-cc66e0f8-77c3-4a44-b876-b167a9d4234e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1a99d8966c14722b49f35d014e203c5', 'request_id': 'req-cc66e0f8-77c3-4a44-b876-b167a9d4234e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:45.963482 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-cc66e0f8-77c3-4a44-b876-b167a9d4234e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1a99d8966c14722b49f35d014e203c5', 'request_id': 'req-cc66e0f8-77c3-4a44-b876-b167a9d4234e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:45.979336 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-cc66e0f8-77c3-4a44-b876-b167a9d4234e admin admin] The token cache is being invalidate because project a1a99d8966c14722b49f35d014e203c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:45.979663 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-cc66e0f8-77c3-4a44-b876-b167a9d4234e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:45.980354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-cc66e0f8-77c3-4a44-b876-b167a9d4234e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:45.981945 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 989/1982] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:45 2022] DELETE /identity/v3/projects/a1a99d8966c14722b49f35d014e203c5 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:05:45.985912 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66e0f8-77c3-4a44-b876-b167a9d4234e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:45.987286 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cc66e0f8-77c3-4a44-b876-b167a9d4234e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:46.007113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d997e4e-877d-48e6-ad2e-045c9b82a3b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:46.007773 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d997e4e-877d-48e6-ad2e-045c9b82a3b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:46.008068 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d997e4e-877d-48e6-ad2e-045c9b82a3b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:46.008348 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d997e4e-877d-48e6-ad2e-045c9b82a3b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:46.010215 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4d997e4e-877d-48e6-ad2e-045c9b82a3b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:46.029749 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:46.035185 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:46.072608 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66e0f8-77c3-4a44-b876-b167a9d4234e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:46.078283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a471bb0-6905-4d1c-97e7-ecd51308eaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:46.078970 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a471bb0-6905-4d1c-97e7-ecd51308eaca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:46.079245 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a471bb0-6905-4d1c-97e7-ecd51308eaca admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:46.079478 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a471bb0-6905-4d1c-97e7-ecd51308eaca admin admin] PATH_INFO: `/v3/users/3ad4f2ea64df4c71994fd9f69811cc55` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:46.080770 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a471bb0-6905-4d1c-97e7-ecd51308eaca admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ad4f2ea64df4c71994fd9f69811cc55)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:46.083207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a471bb0-6905-4d1c-97e7-ecd51308eaca admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:46.093465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d997e4e-877d-48e6-ad2e-045c9b82a3b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:46.095139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d997e4e-877d-48e6-ad2e-045c9b82a3b9 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:46.105662 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 994/1983] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:05:46.137440 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1a471bb0-6905-4d1c-97e7-ecd51308eaca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ad4f2ea64df4c71994fd9f69811cc55'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:46.138108 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1a471bb0-6905-4d1c-97e7-ecd51308eaca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ad4f2ea64df4c71994fd9f69811cc55'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:46.146257 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1a471bb0-6905-4d1c-97e7-ecd51308eaca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ad4f2ea64df4c71994fd9f69811cc55'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:46.146923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1a471bb0-6905-4d1c-97e7-ecd51308eaca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ad4f2ea64df4c71994fd9f69811cc55'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:46.154400 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1a471bb0-6905-4d1c-97e7-ecd51308eaca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ad4f2ea64df4c71994fd9f69811cc55'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:46.179924 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 990/1984] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:45 2022] DELETE /identity/v3/users/3ad4f2ea64df4c71994fd9f69811cc55 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:46.447806 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d997e4e-877d-48e6-ad2e-045c9b82a3b9 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:46.449271 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4d997e4e-877d-48e6-ad2e-045c9b82a3b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:46.470590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d997e4e-877d-48e6-ad2e-045c9b82a3b9 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:46.574337 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faff5cd-dd6d-4e82-aa1a-b589c5244cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:46.575186 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faff5cd-dd6d-4e82-aa1a-b589c5244cc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:46.575586 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faff5cd-dd6d-4e82-aa1a-b589c5244cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:46.577208 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faff5cd-dd6d-4e82-aa1a-b589c5244cc3 admin admin] PATH_INFO: `/v3/projects/d4e64f53fb124fa6b70e35bf156e2545` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:46.579137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faff5cd-dd6d-4e82-aa1a-b589c5244cc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4e64f53fb124fa6b70e35bf156e2545)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:46.581367 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faff5cd-dd6d-4e82-aa1a-b589c5244cc3 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:46.640710 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2faff5cd-dd6d-4e82-aa1a-b589c5244cc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4e64f53fb124fa6b70e35bf156e2545', 'request_id': 'req-2faff5cd-dd6d-4e82-aa1a-b589c5244cc3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:46.649389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2faff5cd-dd6d-4e82-aa1a-b589c5244cc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4e64f53fb124fa6b70e35bf156e2545', 'request_id': 'req-2faff5cd-dd6d-4e82-aa1a-b589c5244cc3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:46.659369 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2faff5cd-dd6d-4e82-aa1a-b589c5244cc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4e64f53fb124fa6b70e35bf156e2545', 'request_id': 'req-2faff5cd-dd6d-4e82-aa1a-b589c5244cc3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:46.671931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a471bb0-6905-4d1c-97e7-ecd51308eaca admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:46.673912 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1a471bb0-6905-4d1c-97e7-ecd51308eaca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:46.680443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2faff5cd-dd6d-4e82-aa1a-b589c5244cc3 admin admin] The token cache is being invalidate because project d4e64f53fb124fa6b70e35bf156e2545 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:46.680864 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2faff5cd-dd6d-4e82-aa1a-b589c5244cc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:46.681609 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2faff5cd-dd6d-4e82-aa1a-b589c5244cc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:46.683872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 995/1985] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:46 2022] DELETE /identity/v3/projects/d4e64f53fb124fa6b70e35bf156e2545 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:46.687575 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faff5cd-dd6d-4e82-aa1a-b589c5244cc3 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:46.688586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a471bb0-6905-4d1c-97e7-ecd51308eaca admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:46.689329 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2faff5cd-dd6d-4e82-aa1a-b589c5244cc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:46.753518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:46.779097 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faff5cd-dd6d-4e82-aa1a-b589c5244cc3 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:46.779755 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9e4bb-aea4-450e-b5cd-8ee82a906f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:46.780405 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9e4bb-aea4-450e-b5cd-8ee82a906f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:46.780900 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9e4bb-aea4-450e-b5cd-8ee82a906f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:46.781287 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9e4bb-aea4-450e-b5cd-8ee82a906f3b admin admin] PATH_INFO: `/v3/projects/83f26fe886fe41f8bd41ef54a54b8675` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:46.782614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9e4bb-aea4-450e-b5cd-8ee82a906f3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=83f26fe886fe41f8bd41ef54a54b8675)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:46.784355 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef69fac-9c9e-4361-8311-1ba06016d5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:46.785068 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef69fac-9c9e-4361-8311-1ba06016d5eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:46.785179 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9e4bb-aea4-450e-b5cd-8ee82a906f3b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:46.785352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef69fac-9c9e-4361-8311-1ba06016d5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:46.785833 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef69fac-9c9e-4361-8311-1ba06016d5eb admin admin] PATH_INFO: `/v3/users/f42bc8658ee74001a5a9179af9b117ab` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:46.786898 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef69fac-9c9e-4361-8311-1ba06016d5eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f42bc8658ee74001a5a9179af9b117ab)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:46.789423 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef69fac-9c9e-4361-8311-1ba06016d5eb admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:46.846292 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-32e9e4bb-aea4-450e-b5cd-8ee82a906f3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83f26fe886fe41f8bd41ef54a54b8675', 'request_id': 'req-32e9e4bb-aea4-450e-b5cd-8ee82a906f3b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:46.850665 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:46.857378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-32e9e4bb-aea4-450e-b5cd-8ee82a906f3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83f26fe886fe41f8bd41ef54a54b8675', 'request_id': 'req-32e9e4bb-aea4-450e-b5cd-8ee82a906f3b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:46.863067 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1ef69fac-9c9e-4361-8311-1ba06016d5eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f42bc8658ee74001a5a9179af9b117ab'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:46.865177 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1ef69fac-9c9e-4361-8311-1ba06016d5eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f42bc8658ee74001a5a9179af9b117ab'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:46.878126 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1ef69fac-9c9e-4361-8311-1ba06016d5eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f42bc8658ee74001a5a9179af9b117ab'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:46.880232 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-32e9e4bb-aea4-450e-b5cd-8ee82a906f3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83f26fe886fe41f8bd41ef54a54b8675', 'request_id': 'req-32e9e4bb-aea4-450e-b5cd-8ee82a906f3b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:46.888284 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1ef69fac-9c9e-4361-8311-1ba06016d5eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f42bc8658ee74001a5a9179af9b117ab'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:46.889065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1ef69fac-9c9e-4361-8311-1ba06016d5eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f42bc8658ee74001a5a9179af9b117ab'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:46.898708 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-32e9e4bb-aea4-450e-b5cd-8ee82a906f3b admin admin] The token cache is being invalidate because project 83f26fe886fe41f8bd41ef54a54b8675 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:46.899145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-32e9e4bb-aea4-450e-b5cd-8ee82a906f3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:46.900860 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-32e9e4bb-aea4-450e-b5cd-8ee82a906f3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:46.907926 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 991/1986] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:46 2022] DELETE /identity/v3/projects/83f26fe886fe41f8bd41ef54a54b8675 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:05:46.910058 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eae4d9-6bd2-46c9-8116-1aafad8a4999 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:46.910356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eae4d9-6bd2-46c9-8116-1aafad8a4999 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:46.910986 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eae4d9-6bd2-46c9-8116-1aafad8a4999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:46.916787 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 996/1987] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:46 2022] DELETE /identity/v3/users/f42bc8658ee74001a5a9179af9b117ab => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:05:46.942509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f2eae4d9-6bd2-46c9-8116-1aafad8a4999 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:46.944656 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f2eae4d9-6bd2-46c9-8116-1aafad8a4999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:47.057379 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 992/1988] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:47.062166 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b4316-39f1-49f0-a728-93589d562786 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:47.062455 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b4316-39f1-49f0-a728-93589d562786 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:47.062742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b4316-39f1-49f0-a728-93589d562786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:47.094616 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-7d5b4316-39f1-49f0-a728-93589d562786 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:47.095926 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7d5b4316-39f1-49f0-a728-93589d562786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:47.104531 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 997/1989] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:47.109254 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eae4d9-6bd2-46c9-8116-1aafad8a4999 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:47.129276 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eae4d9-6bd2-46c9-8116-1aafad8a4999 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:47.135475 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26b335-5a2b-4742-95c5-6d87e59ddc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:47.136189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26b335-5a2b-4742-95c5-6d87e59ddc98 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:47.136478 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26b335-5a2b-4742-95c5-6d87e59ddc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:47.136755 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb26b335-5a2b-4742-95c5-6d87e59ddc98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:47.137558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26b335-5a2b-4742-95c5-6d87e59ddc98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:05:47.137870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26b335-5a2b-4742-95c5-6d87e59ddc98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:47.140033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb26b335-5a2b-4742-95c5-6d87e59ddc98 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:47.156618 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 993/1990] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:05:47.161172 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b4316-39f1-49f0-a728-93589d562786 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:47.175243 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b4316-39f1-49f0-a728-93589d562786 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:47.195867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b58ba-415d-43e1-8deb-6dda3459aa71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:47.195867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b58ba-415d-43e1-8deb-6dda3459aa71 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:47.195867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b58ba-415d-43e1-8deb-6dda3459aa71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:47.195867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b58ba-415d-43e1-8deb-6dda3459aa71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:47.196394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b58ba-415d-43e1-8deb-6dda3459aa71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:47.196394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b58ba-415d-43e1-8deb-6dda3459aa71 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:47.232359 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 998/1991] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:05:47 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:47.233839 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26b335-5a2b-4742-95c5-6d87e59ddc98 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:47.248575 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb26b335-5a2b-4742-95c5-6d87e59ddc98 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:47.284549 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b58ba-415d-43e1-8deb-6dda3459aa71 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:47.287116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-fb9b58ba-415d-43e1-8deb-6dda3459aa71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:47.318918 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe39a56-115d-4516-88da-bc1fc3bc0b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:47.319644 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe39a56-115d-4516-88da-bc1fc3bc0b33 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:47.319987 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe39a56-115d-4516-88da-bc1fc3bc0b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:47.320272 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe39a56-115d-4516-88da-bc1fc3bc0b33 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:47.321038 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe39a56-115d-4516-88da-bc1fc3bc0b33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:47.323116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe39a56-115d-4516-88da-bc1fc3bc0b33 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:47.347472 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 994/1992] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:05:47 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:05:47.352028 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe39a56-115d-4516-88da-bc1fc3bc0b33 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:47.362720 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe39a56-115d-4516-88da-bc1fc3bc0b33 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:47.366832 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b58ba-415d-43e1-8deb-6dda3459aa71 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:47.368242 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27c4e5-d947-4dc6-b47f-5d4977d9ffda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:47.368859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27c4e5-d947-4dc6-b47f-5d4977d9ffda admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:47.369167 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27c4e5-d947-4dc6-b47f-5d4977d9ffda admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:47.369444 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27c4e5-d947-4dc6-b47f-5d4977d9ffda admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:47.370091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa27c4e5-d947-4dc6-b47f-5d4977d9ffda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:47.371500 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4532b22-e538-4d2c-98ff-e6034ae7e8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:47.371743 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa27c4e5-d947-4dc6-b47f-5d4977d9ffda admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:47.372155 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4532b22-e538-4d2c-98ff-e6034ae7e8df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:47.372465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4532b22-e538-4d2c-98ff-e6034ae7e8df admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:47.372746 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4532b22-e538-4d2c-98ff-e6034ae7e8df admin admin] PATH_INFO: `/v3/projects/41e271f4396b43d59c2493b6fe83ebe8` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:47.373883 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4532b22-e538-4d2c-98ff-e6034ae7e8df admin admin] RBAC: Authorizing `identity:delete_project(project_id=41e271f4396b43d59c2493b6fe83ebe8)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:47.375877 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4532b22-e538-4d2c-98ff-e6034ae7e8df admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:47.383668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 995/1993] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:05:47.387762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27c4e5-d947-4dc6-b47f-5d4977d9ffda admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:47.399187 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27c4e5-d947-4dc6-b47f-5d4977d9ffda admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:47.410876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90483e-e8c9-4ba7-b784-167de0ef3bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:47.410876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90483e-e8c9-4ba7-b784-167de0ef3bba admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:47.410876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90483e-e8c9-4ba7-b784-167de0ef3bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:47.410876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90483e-e8c9-4ba7-b784-167de0ef3bba admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:47.411431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90483e-e8c9-4ba7-b784-167de0ef3bba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:47.411431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f90483e-e8c9-4ba7-b784-167de0ef3bba admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:47.424767 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 996/1994] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:05:47.424970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:47.426150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90483e-e8c9-4ba7-b784-167de0ef3bba admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:47.434957 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c4532b22-e538-4d2c-98ff-e6034ae7e8df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41e271f4396b43d59c2493b6fe83ebe8', 'request_id': 'req-c4532b22-e538-4d2c-98ff-e6034ae7e8df'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:47.437756 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90483e-e8c9-4ba7-b784-167de0ef3bba admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:47.443851 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c4532b22-e538-4d2c-98ff-e6034ae7e8df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41e271f4396b43d59c2493b6fe83ebe8', 'request_id': 'req-c4532b22-e538-4d2c-98ff-e6034ae7e8df'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:47.449712 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c4532b22-e538-4d2c-98ff-e6034ae7e8df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41e271f4396b43d59c2493b6fe83ebe8', 'request_id': 'req-c4532b22-e538-4d2c-98ff-e6034ae7e8df'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:47.461501 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c4532b22-e538-4d2c-98ff-e6034ae7e8df admin admin] The token cache is being invalidate because project 41e271f4396b43d59c2493b6fe83ebe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:47.461792 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c4532b22-e538-4d2c-98ff-e6034ae7e8df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:47.462461 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c4532b22-e538-4d2c-98ff-e6034ae7e8df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:47.464120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 999/1995] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:47 2022] DELETE /identity/v3/projects/41e271f4396b43d59c2493b6fe83ebe8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:05:47.470265 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fdafce-cf9a-421e-97a6-3c632e347bcd None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:47.470915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fdafce-cf9a-421e-97a6-3c632e347bcd None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:47.471591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fdafce-cf9a-421e-97a6-3c632e347bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:47.490947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:47.495990 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-54fdafce-cf9a-421e-97a6-3c632e347bcd None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:47.497311 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-54fdafce-cf9a-421e-97a6-3c632e347bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:47.524953 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af4ed8-75c9-4b0c-8713-eec06f2ba9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:47.525972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af4ed8-75c9-4b0c-8713-eec06f2ba9ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:47.526594 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af4ed8-75c9-4b0c-8713-eec06f2ba9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:47.527116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af4ed8-75c9-4b0c-8713-eec06f2ba9ba admin admin] PATH_INFO: `/v3/projects/6df5fe3a39fd4f9fb1ae104e43ec5529/users/9bca70e61b80499eb07a668e11534257/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:47.550606 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af4ed8-75c9-4b0c-8713-eec06f2ba9ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=6df5fe3a39fd4f9fb1ae104e43ec5529, user_id=9bca70e61b80499eb07a668e11534257, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:47.561384 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af4ed8-75c9-4b0c-8713-eec06f2ba9ba admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:47.565541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1000/1996] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:05:47.573833 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab263832-c208-4142-9f29-ddb7694d517d None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:47.574124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab263832-c208-4142-9f29-ddb7694d517d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:47.574407 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab263832-c208-4142-9f29-ddb7694d517d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:47.596331 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 997/1997] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:47 2022] PUT /identity/v3/projects/6df5fe3a39fd4f9fb1ae104e43ec5529/users/9bca70e61b80499eb07a668e11534257/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:05:47.602543 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-ab263832-c208-4142-9f29-ddb7694d517d None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:47.602934 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c0ca8-4017-464b-9b57-96bbf6bf7a68 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:47.604902 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c0ca8-4017-464b-9b57-96bbf6bf7a68 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:47.604902 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c0ca8-4017-464b-9b57-96bbf6bf7a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:47.609072 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ab263832-c208-4142-9f29-ddb7694d517d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:47.649820 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-334c0ca8-4017-464b-9b57-96bbf6bf7a68 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:47.651311 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-334c0ca8-4017-464b-9b57-96bbf6bf7a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:47.703794 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1001/1998] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:47.704756 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab263832-c208-4142-9f29-ddb7694d517d None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:47.714356 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab263832-c208-4142-9f29-ddb7694d517d None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:47.719191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ced6a-ef8a-4cbe-865f-7d1aa83d7e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:47.719812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ced6a-ef8a-4cbe-865f-7d1aa83d7e63 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:47.720097 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ced6a-ef8a-4cbe-865f-7d1aa83d7e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:47.720376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ced6a-ef8a-4cbe-865f-7d1aa83d7e63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:47.721029 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ced6a-ef8a-4cbe-865f-7d1aa83d7e63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:05:47.721336 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ced6a-ef8a-4cbe-865f-7d1aa83d7e63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:47.722954 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ced6a-ef8a-4cbe-865f-7d1aa83d7e63 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:47.727542 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 998/1999] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:47.735514 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c0ca8-4017-464b-9b57-96bbf6bf7a68 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:47.736927 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-334c0ca8-4017-464b-9b57-96bbf6bf7a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:47.737899 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1002/2000] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:05:47.743049 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ced6a-ef8a-4cbe-865f-7d1aa83d7e63 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:47.755923 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ced6a-ef8a-4cbe-865f-7d1aa83d7e63 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:47.762373 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb388a97-77fd-4107-8c37-774f2c93c9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:47.763637 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb388a97-77fd-4107-8c37-774f2c93c9ce admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:47.763637 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb388a97-77fd-4107-8c37-774f2c93c9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:47.763838 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb388a97-77fd-4107-8c37-774f2c93c9ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:47.764665 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb388a97-77fd-4107-8c37-774f2c93c9ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:47.766773 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb388a97-77fd-4107-8c37-774f2c93c9ce admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:47.804558 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1003/2001] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:05:47 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:47.805392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c0ca8-4017-464b-9b57-96bbf6bf7a68 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:47.809852 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb388a97-77fd-4107-8c37-774f2c93c9ce admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:47.858096 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:47.885154 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39b6e9-e5a6-426d-b03d-b82e5b4a9e78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:47.885785 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39b6e9-e5a6-426d-b03d-b82e5b4a9e78 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:47.886076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39b6e9-e5a6-426d-b03d-b82e5b4a9e78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:47.886345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39b6e9-e5a6-426d-b03d-b82e5b4a9e78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:47.890093 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb388a97-77fd-4107-8c37-774f2c93c9ce admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:47.895491 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33774d-f963-4ce3-a894-e0c689533b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:47.896266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33774d-f963-4ce3-a894-e0c689533b40 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:47.896552 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33774d-f963-4ce3-a894-e0c689533b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:47.896829 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33774d-f963-4ce3-a894-e0c689533b40 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:47.896943 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39b6e9-e5a6-426d-b03d-b82e5b4a9e78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:47.897506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33774d-f963-4ce3-a894-e0c689533b40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:47.898449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39b6e9-e5a6-426d-b03d-b82e5b4a9e78 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:47.899445 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33774d-f963-4ce3-a894-e0c689533b40 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:47.911493 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 999/2002] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:05:47.923383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1004/2003] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:05:47 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:47.927851 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39b6e9-e5a6-426d-b03d-b82e5b4a9e78 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:47.937606 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39b6e9-e5a6-426d-b03d-b82e5b4a9e78 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:47.942631 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfc68b-4683-449d-9f10-de3ff90f3db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:47.943320 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfc68b-4683-449d-9f10-de3ff90f3db5 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:47.943631 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfc68b-4683-449d-9f10-de3ff90f3db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:47.944039 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfc68b-4683-449d-9f10-de3ff90f3db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:47.944705 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dfc68b-4683-449d-9f10-de3ff90f3db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:47.946398 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dfc68b-4683-449d-9f10-de3ff90f3db5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:47.957292 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1000/2004] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:47.962519 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33774d-f963-4ce3-a894-e0c689533b40 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:47.975018 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33774d-f963-4ce3-a894-e0c689533b40 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:47.979861 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ac95d-91d2-4aad-9ec3-5b4f5bf3c087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:47.980480 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ac95d-91d2-4aad-9ec3-5b4f5bf3c087 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:47.980764 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ac95d-91d2-4aad-9ec3-5b4f5bf3c087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:47.981037 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ac95d-91d2-4aad-9ec3-5b4f5bf3c087 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:47.981713 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ac95d-91d2-4aad-9ec3-5b4f5bf3c087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:47.983767 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ac95d-91d2-4aad-9ec3-5b4f5bf3c087 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:47.994587 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1005/2005] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:47.998668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfc68b-4683-449d-9f10-de3ff90f3db5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:48.009034 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfc68b-4683-449d-9f10-de3ff90f3db5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:48.013778 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c60fe-dc31-4bc6-aad7-78592d18e147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:48.014702 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c60fe-dc31-4bc6-aad7-78592d18e147 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:48.014987 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c60fe-dc31-4bc6-aad7-78592d18e147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:48.015263 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c60fe-dc31-4bc6-aad7-78592d18e147 admin admin] PATH_INFO: `/v3/projects/f1d326e5bd894aeb8fa6519ef115861c/users/dea24093fdaa4e1da1ea7696f86d4d53/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:48.018076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:48.048108 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c60fe-dc31-4bc6-aad7-78592d18e147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1d326e5bd894aeb8fa6519ef115861c, user_id=dea24093fdaa4e1da1ea7696f86d4d53, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:48.050880 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c60fe-dc31-4bc6-aad7-78592d18e147 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:48.072661 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1001/2006] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:47 2022] PUT /identity/v3/projects/f1d326e5bd894aeb8fa6519ef115861c/users/dea24093fdaa4e1da1ea7696f86d4d53/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:48.084426 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6372e3-d375-4b2a-abfd-a186f09e5147 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:48.084782 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6372e3-d375-4b2a-abfd-a186f09e5147 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:48.085003 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6372e3-d375-4b2a-abfd-a186f09e5147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:48.117673 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-1e6372e3-d375-4b2a-abfd-a186f09e5147 None None] MFA Rules not processed for user `dea24093fdaa4e1da1ea7696f86d4d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:48.119344 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1e6372e3-d375-4b2a-abfd-a186f09e5147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:48.250383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1006/2007] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:48.259018 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c60fe-dc31-4bc6-aad7-78592d18e147 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:48.366755 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c60fe-dc31-4bc6-aad7-78592d18e147 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:48.372953 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec0d8d-7988-4116-9fb5-9e099090052b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:48.374090 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec0d8d-7988-4116-9fb5-9e099090052b admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:48.374258 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec0d8d-7988-4116-9fb5-9e099090052b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:48.374532 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec0d8d-7988-4116-9fb5-9e099090052b admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:48.375271 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec0d8d-7988-4116-9fb5-9e099090052b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:48.378233 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec0d8d-7988-4116-9fb5-9e099090052b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:48.418573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1002/2008] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:05:48 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:48.419948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6372e3-d375-4b2a-abfd-a186f09e5147 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:48.434773 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6372e3-d375-4b2a-abfd-a186f09e5147 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:48.541770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ef05a-ffd5-43be-a355-4d8b76285051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:48.542790 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ef05a-ffd5-43be-a355-4d8b76285051 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:48.543284 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ef05a-ffd5-43be-a355-4d8b76285051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:48.543899 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ef05a-ffd5-43be-a355-4d8b76285051 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:48.544803 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ef05a-ffd5-43be-a355-4d8b76285051 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:48.551819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ef05a-ffd5-43be-a355-4d8b76285051 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:48.580479 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1007/2009] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:05:48 2022] POST /identity/v3/users => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:48.584558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec0d8d-7988-4116-9fb5-9e099090052b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:48.612581 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec0d8d-7988-4116-9fb5-9e099090052b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:48.617522 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80f72d-cc69-4235-ad4f-b57d4256d242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:48.618148 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80f72d-cc69-4235-ad4f-b57d4256d242 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:48.618456 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80f72d-cc69-4235-ad4f-b57d4256d242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:48.618709 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80f72d-cc69-4235-ad4f-b57d4256d242 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:48.619358 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f80f72d-cc69-4235-ad4f-b57d4256d242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:48.621158 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f80f72d-cc69-4235-ad4f-b57d4256d242 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:48.637006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1003/2010] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:48.641509 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ef05a-ffd5-43be-a355-4d8b76285051 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:48.653479 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ef05a-ffd5-43be-a355-4d8b76285051 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:48.661728 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85776945-30e4-4af5-9286-cf3f7761ed10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:48.662972 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85776945-30e4-4af5-9286-cf3f7761ed10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:48.662972 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85776945-30e4-4af5-9286-cf3f7761ed10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:48.669659 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85776945-30e4-4af5-9286-cf3f7761ed10 admin admin] PATH_INFO: `/v3/projects/440c3550f6764cf5a76367a8ea2b0a9e/users/4c2c389da4a94cfebbb85311791cd112/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:48.708542 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85776945-30e4-4af5-9286-cf3f7761ed10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=440c3550f6764cf5a76367a8ea2b0a9e, user_id=4c2c389da4a94cfebbb85311791cd112, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:48.723596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85776945-30e4-4af5-9286-cf3f7761ed10 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:48.735303 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1008/2011] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:48 2022] PUT /identity/v3/projects/440c3550f6764cf5a76367a8ea2b0a9e/users/4c2c389da4a94cfebbb85311791cd112/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:05:48.740516 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80f72d-cc69-4235-ad4f-b57d4256d242 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:48.751136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80f72d-cc69-4235-ad4f-b57d4256d242 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:48.812745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd264f60-331e-4c67-ae12-f2f3b983a576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:48.813370 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd264f60-331e-4c67-ae12-f2f3b983a576 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:48.813655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd264f60-331e-4c67-ae12-f2f3b983a576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:48.813929 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd264f60-331e-4c67-ae12-f2f3b983a576 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:48.814578 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd264f60-331e-4c67-ae12-f2f3b983a576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:48.816255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd264f60-331e-4c67-ae12-f2f3b983a576 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:48.831443 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1004/2012] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:48.836457 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85776945-30e4-4af5-9286-cf3f7761ed10 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:48.847914 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85776945-30e4-4af5-9286-cf3f7761ed10 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:48.853082 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e474b-3716-46da-bdcd-3f9cff2b2da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:48.854000 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e474b-3716-46da-bdcd-3f9cff2b2da3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:48.854368 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e474b-3716-46da-bdcd-3f9cff2b2da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:48.854708 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e474b-3716-46da-bdcd-3f9cff2b2da3 admin admin] PATH_INFO: `/v3/projects/440c3550f6764cf5a76367a8ea2b0a9e/users/4c2c389da4a94cfebbb85311791cd112/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:48.856766 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e474b-3716-46da-bdcd-3f9cff2b2da3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=440c3550f6764cf5a76367a8ea2b0a9e, user_id=4c2c389da4a94cfebbb85311791cd112, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:48.859623 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e474b-3716-46da-bdcd-3f9cff2b2da3 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:48.862158 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:48.879609 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1009/2013] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:48 2022] PUT /identity/v3/projects/440c3550f6764cf5a76367a8ea2b0a9e/users/4c2c389da4a94cfebbb85311791cd112/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:05:48.885283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc0dd4-15fe-4142-85c2-c261f0e8bfdb None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:48.885569 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc0dd4-15fe-4142-85c2-c261f0e8bfdb None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:48.885847 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc0dd4-15fe-4142-85c2-c261f0e8bfdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:48.919382 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c1fc0dd4-15fe-4142-85c2-c261f0e8bfdb None None] MFA Rules not processed for user `4c2c389da4a94cfebbb85311791cd112`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:48.920659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c1fc0dd4-15fe-4142-85c2-c261f0e8bfdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:48.971031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:49.009465 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1005/2014] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:48 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:49.019227 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e474b-3716-46da-bdcd-3f9cff2b2da3 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:49.022164 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-de3e474b-3716-46da-bdcd-3f9cff2b2da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:49.153598 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e474b-3716-46da-bdcd-3f9cff2b2da3 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:49.157794 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd0f96-904b-4cfa-8c35-26276e962781 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:49.159096 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd0f96-904b-4cfa-8c35-26276e962781 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:49.159096 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd0f96-904b-4cfa-8c35-26276e962781 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:49.159291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd0f96-904b-4cfa-8c35-26276e962781 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:49.180666 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd0f96-904b-4cfa-8c35-26276e962781 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:49.182806 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd0f96-904b-4cfa-8c35-26276e962781 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:49.236270 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1010/2015] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:49 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:49.360931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc0dd4-15fe-4142-85c2-c261f0e8bfdb None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:49.374623 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc0dd4-15fe-4142-85c2-c261f0e8bfdb None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:49.381479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2951f-5b54-425b-9b62-b0268967d5ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:49.382339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2951f-5b54-425b-9b62-b0268967d5ba service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:49.382744 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2951f-5b54-425b-9b62-b0268967d5ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:49.383186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2951f-5b54-425b-9b62-b0268967d5ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:49.497797 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd2951f-5b54-425b-9b62-b0268967d5ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:49.499518 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd2951f-5b54-425b-9b62-b0268967d5ba service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:49.513233 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1006/2016] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:49.744729 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb3f00-76e8-40a2-9cc2-d0ce8a857f5a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:49.745015 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb3f00-76e8-40a2-9cc2-d0ce8a857f5a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:49.745317 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb3f00-76e8-40a2-9cc2-d0ce8a857f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:49.835392 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-edcb3f00-76e8-40a2-9cc2-d0ce8a857f5a None None] MFA Rules not processed for user `9bca70e61b80499eb07a668e11534257`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:49.836667 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-edcb3f00-76e8-40a2-9cc2-d0ce8a857f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:49.928265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2951f-5b54-425b-9b62-b0268967d5ba service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:49.943438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1011/2017] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:49.959352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb3f00-76e8-40a2-9cc2-d0ce8a857f5a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:49.971308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-edcb3f00-76e8-40a2-9cc2-d0ce8a857f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:50.015444 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb3f00-76e8-40a2-9cc2-d0ce8a857f5a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:50.027705 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf3452-fdeb-42d4-960f-50e0b93d70b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:50.028777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf3452-fdeb-42d4-960f-50e0b93d70b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:50.029211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf3452-fdeb-42d4-960f-50e0b93d70b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:50.029645 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf3452-fdeb-42d4-960f-50e0b93d70b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:50.042817 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf3452-fdeb-42d4-960f-50e0b93d70b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:50.044343 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf3452-fdeb-42d4-960f-50e0b93d70b7 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:50.062412 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1012/2018] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:50.109996 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2951f-5b54-425b-9b62-b0268967d5ba service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:50.133040 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60587b0-4ed6-4b85-b86e-5c6e5a123224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:50.133754 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60587b0-4ed6-4b85-b86e-5c6e5a123224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:50.134039 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60587b0-4ed6-4b85-b86e-5c6e5a123224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:50.134320 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60587b0-4ed6-4b85-b86e-5c6e5a123224 admin admin] PATH_INFO: `/v3/users/dea24093fdaa4e1da1ea7696f86d4d53` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:50.137272 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60587b0-4ed6-4b85-b86e-5c6e5a123224 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dea24093fdaa4e1da1ea7696f86d4d53)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:50.140603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60587b0-4ed6-4b85-b86e-5c6e5a123224 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:50.232148 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b60587b0-4ed6-4b85-b86e-5c6e5a123224 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dea24093fdaa4e1da1ea7696f86d4d53'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:50.233709 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b60587b0-4ed6-4b85-b86e-5c6e5a123224 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dea24093fdaa4e1da1ea7696f86d4d53'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:50.246808 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b60587b0-4ed6-4b85-b86e-5c6e5a123224 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dea24093fdaa4e1da1ea7696f86d4d53'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:50.247838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b60587b0-4ed6-4b85-b86e-5c6e5a123224 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dea24093fdaa4e1da1ea7696f86d4d53'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:50.257403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b60587b0-4ed6-4b85-b86e-5c6e5a123224 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dea24093fdaa4e1da1ea7696f86d4d53'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:50.273567 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:50.288036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1007/2019] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:49 2022] DELETE /identity/v3/users/dea24093fdaa4e1da1ea7696f86d4d53 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:05:50.294513 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b761ea-998a-4068-89fb-72067f0d46d2 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:50.294844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b761ea-998a-4068-89fb-72067f0d46d2 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:50.295087 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b761ea-998a-4068-89fb-72067f0d46d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:50.319947 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-51b761ea-998a-4068-89fb-72067f0d46d2 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:50.336898 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-51b761ea-998a-4068-89fb-72067f0d46d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:50.517724 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1013/2020] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:50.540859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60587b0-4ed6-4b85-b86e-5c6e5a123224 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:50.543982 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b60587b0-4ed6-4b85-b86e-5c6e5a123224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:50.746345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60587b0-4ed6-4b85-b86e-5c6e5a123224 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:50.752261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffaf33-8de3-4170-9188-697c2758aed4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:50.753909 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffaf33-8de3-4170-9188-697c2758aed4 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:50.755258 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffaf33-8de3-4170-9188-697c2758aed4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:50.755818 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffaf33-8de3-4170-9188-697c2758aed4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:50.767819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ffaf33-8de3-4170-9188-697c2758aed4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:50.769450 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ffaf33-8de3-4170-9188-697c2758aed4 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:50.799437 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1008/2021] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:51.494063 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b761ea-998a-4068-89fb-72067f0d46d2 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:51.496300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-51b761ea-998a-4068-89fb-72067f0d46d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:51.516106 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b761ea-998a-4068-89fb-72067f0d46d2 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:51.528201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f05f19a-88a1-4d24-ba92-ab62ea061e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:51.529191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05f19a-88a1-4d24-ba92-ab62ea061e18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:51.529435 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05f19a-88a1-4d24-ba92-ab62ea061e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:51.529770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05f19a-88a1-4d24-ba92-ab62ea061e18 admin admin] PATH_INFO: `/v3/projects/f1d326e5bd894aeb8fa6519ef115861c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:51.533596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f05f19a-88a1-4d24-ba92-ab62ea061e18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1d326e5bd894aeb8fa6519ef115861c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:51.535727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f05f19a-88a1-4d24-ba92-ab62ea061e18 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:51.570141 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:51.659716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8f05f19a-88a1-4d24-ba92-ab62ea061e18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1d326e5bd894aeb8fa6519ef115861c', 'request_id': 'req-8f05f19a-88a1-4d24-ba92-ab62ea061e18'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:51.671506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8f05f19a-88a1-4d24-ba92-ab62ea061e18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1d326e5bd894aeb8fa6519ef115861c', 'request_id': 'req-8f05f19a-88a1-4d24-ba92-ab62ea061e18'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:51.679257 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8f05f19a-88a1-4d24-ba92-ab62ea061e18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1d326e5bd894aeb8fa6519ef115861c', 'request_id': 'req-8f05f19a-88a1-4d24-ba92-ab62ea061e18'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:51.692618 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8f05f19a-88a1-4d24-ba92-ab62ea061e18 admin admin] The token cache is being invalidate because project f1d326e5bd894aeb8fa6519ef115861c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:51.693166 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8f05f19a-88a1-4d24-ba92-ab62ea061e18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:51.693736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8f05f19a-88a1-4d24-ba92-ab62ea061e18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:51.695683 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1014/2022] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:51 2022] DELETE /identity/v3/projects/f1d326e5bd894aeb8fa6519ef115861c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:05:51.699598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffaf33-8de3-4170-9188-697c2758aed4 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:51.710523 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffaf33-8de3-4170-9188-697c2758aed4 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:51.798629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f1d02-194b-40e2-ba31-8d168b2637f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:51.799344 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f1d02-194b-40e2-ba31-8d168b2637f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:51.799840 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f1d02-194b-40e2-ba31-8d168b2637f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:51.800263 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f1d02-194b-40e2-ba31-8d168b2637f1 admin admin] PATH_INFO: `/v3/users/4c2c389da4a94cfebbb85311791cd112` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:51.802437 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f1d02-194b-40e2-ba31-8d168b2637f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c2c389da4a94cfebbb85311791cd112)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:51.804594 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f1d02-194b-40e2-ba31-8d168b2637f1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:51.866213 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0acfb6-8090-40d3-9f3b-2ac80774300f None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:51.866556 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0acfb6-8090-40d3-9f3b-2ac80774300f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:51.866768 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0acfb6-8090-40d3-9f3b-2ac80774300f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:51.897023 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-ed0acfb6-8090-40d3-9f3b-2ac80774300f None None] MFA Rules not processed for user `9bca70e61b80499eb07a668e11534257`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:51.898180 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ed0acfb6-8090-40d3-9f3b-2ac80774300f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:51.908509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f22f1d02-194b-40e2-ba31-8d168b2637f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c2c389da4a94cfebbb85311791cd112'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:51.909206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f22f1d02-194b-40e2-ba31-8d168b2637f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c2c389da4a94cfebbb85311791cd112'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:51.923244 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f22f1d02-194b-40e2-ba31-8d168b2637f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c2c389da4a94cfebbb85311791cd112'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:51.923897 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f22f1d02-194b-40e2-ba31-8d168b2637f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c2c389da4a94cfebbb85311791cd112'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:51.939797 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f22f1d02-194b-40e2-ba31-8d168b2637f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c2c389da4a94cfebbb85311791cd112'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:51.978780 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1009/2023] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:51 2022] DELETE /identity/v3/users/4c2c389da4a94cfebbb85311791cd112 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:05:52.010408 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1015/2024] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:53.010009 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f1d02-194b-40e2-ba31-8d168b2637f1 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:53.012319 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f22f1d02-194b-40e2-ba31-8d168b2637f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:53.072087 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:53.147170 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f1d02-194b-40e2-ba31-8d168b2637f1 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:53.152822 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03fd62-0997-43fe-afd8-7beac8c6823b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:53.153576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03fd62-0997-43fe-afd8-7beac8c6823b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:53.153859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03fd62-0997-43fe-afd8-7beac8c6823b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:53.154183 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03fd62-0997-43fe-afd8-7beac8c6823b admin admin] PATH_INFO: `/v3/projects/440c3550f6764cf5a76367a8ea2b0a9e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:53.155508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03fd62-0997-43fe-afd8-7beac8c6823b admin admin] RBAC: Authorizing `identity:delete_project(project_id=440c3550f6764cf5a76367a8ea2b0a9e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:53.158063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03fd62-0997-43fe-afd8-7beac8c6823b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:53.240834 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4f03fd62-0997-43fe-afd8-7beac8c6823b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '440c3550f6764cf5a76367a8ea2b0a9e', 'request_id': 'req-4f03fd62-0997-43fe-afd8-7beac8c6823b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:53.267150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4f03fd62-0997-43fe-afd8-7beac8c6823b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '440c3550f6764cf5a76367a8ea2b0a9e', 'request_id': 'req-4f03fd62-0997-43fe-afd8-7beac8c6823b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:53.283425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4f03fd62-0997-43fe-afd8-7beac8c6823b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '440c3550f6764cf5a76367a8ea2b0a9e', 'request_id': 'req-4f03fd62-0997-43fe-afd8-7beac8c6823b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:53.315067 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4f03fd62-0997-43fe-afd8-7beac8c6823b admin admin] The token cache is being invalidate because project 440c3550f6764cf5a76367a8ea2b0a9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:53.315657 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4f03fd62-0997-43fe-afd8-7beac8c6823b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:53.316466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4f03fd62-0997-43fe-afd8-7beac8c6823b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:53.323317 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1010/2025] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:53 2022] DELETE /identity/v3/projects/440c3550f6764cf5a76367a8ea2b0a9e => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:53.329307 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b59db-7c6c-4de9-83ba-d82b549b184a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:53.333929 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b59db-7c6c-4de9-83ba-d82b549b184a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:53.334399 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b59db-7c6c-4de9-83ba-d82b549b184a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:53.382552 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-464b59db-7c6c-4de9-83ba-d82b549b184a None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:53.383862 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-464b59db-7c6c-4de9-83ba-d82b549b184a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:53.478525 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1016/2026] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:53.485793 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a837791-086f-4873-b3e7-63737ac7a0fe None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:53.486057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a837791-086f-4873-b3e7-63737ac7a0fe None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:53.486332 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a837791-086f-4873-b3e7-63737ac7a0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:53.525438 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7a837791-086f-4873-b3e7-63737ac7a0fe None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:53.527349 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7a837791-086f-4873-b3e7-63737ac7a0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:53.542728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1011/2027] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:53.544278 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b59db-7c6c-4de9-83ba-d82b549b184a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:53.562705 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b59db-7c6c-4de9-83ba-d82b549b184a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:53.574265 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407df7f-257d-451f-bc42-a05ec2b8678a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:53.575155 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407df7f-257d-451f-bc42-a05ec2b8678a admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:53.575626 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407df7f-257d-451f-bc42-a05ec2b8678a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:53.575971 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407df7f-257d-451f-bc42-a05ec2b8678a admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:53.576778 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407df7f-257d-451f-bc42-a05ec2b8678a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:05:53.577181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407df7f-257d-451f-bc42-a05ec2b8678a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:53.579260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407df7f-257d-451f-bc42-a05ec2b8678a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:53.600370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1017/2028] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:05:53.605188 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a837791-086f-4873-b3e7-63737ac7a0fe None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:53.622241 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a837791-086f-4873-b3e7-63737ac7a0fe None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:53.627669 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba262f-0854-4c5d-9c11-802d9f2c5da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:53.628106 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba262f-0854-4c5d-9c11-802d9f2c5da5 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:53.628394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba262f-0854-4c5d-9c11-802d9f2c5da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:53.628672 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba262f-0854-4c5d-9c11-802d9f2c5da5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:53.629897 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba262f-0854-4c5d-9c11-802d9f2c5da5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:53.632560 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba262f-0854-4c5d-9c11-802d9f2c5da5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:53.701746 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1012/2029] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:05:53 2022] POST /identity/v3/projects => generated 373 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:53.704689 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407df7f-257d-451f-bc42-a05ec2b8678a admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:53.719421 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407df7f-257d-451f-bc42-a05ec2b8678a admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:53.777944 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:53.856613 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854f5d3-50fc-4d0c-8e2f-5fac8739c540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:53.857360 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854f5d3-50fc-4d0c-8e2f-5fac8739c540 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:53.857649 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854f5d3-50fc-4d0c-8e2f-5fac8739c540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:53.858041 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854f5d3-50fc-4d0c-8e2f-5fac8739c540 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:53.859150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0854f5d3-50fc-4d0c-8e2f-5fac8739c540 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:53.861087 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0854f5d3-50fc-4d0c-8e2f-5fac8739c540 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:53.924034 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1018/2030] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:05:53 2022] POST /identity/v3/users => generated 287 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:53.928338 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba262f-0854-4c5d-9c11-802d9f2c5da5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:53.943605 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba262f-0854-4c5d-9c11-802d9f2c5da5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:53.953106 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31734b2c-9ada-4535-bae1-86683767ee95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:53.953828 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31734b2c-9ada-4535-bae1-86683767ee95 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:53.956117 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31734b2c-9ada-4535-bae1-86683767ee95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:53.957116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31734b2c-9ada-4535-bae1-86683767ee95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:53.958775 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31734b2c-9ada-4535-bae1-86683767ee95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:53.962357 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31734b2c-9ada-4535-bae1-86683767ee95 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:54.002453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1013/2031] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:05:54.007371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854f5d3-50fc-4d0c-8e2f-5fac8739c540 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:54.024457 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854f5d3-50fc-4d0c-8e2f-5fac8739c540 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:54.035148 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929059f9-9462-4040-bda0-cd453b7543d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:54.036877 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929059f9-9462-4040-bda0-cd453b7543d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:54.037377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929059f9-9462-4040-bda0-cd453b7543d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:54.037966 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929059f9-9462-4040-bda0-cd453b7543d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:54.039136 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929059f9-9462-4040-bda0-cd453b7543d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:54.044941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929059f9-9462-4040-bda0-cd453b7543d5 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:54.068273 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1019/2032] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:54.073081 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31734b2c-9ada-4535-bae1-86683767ee95 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:54.086620 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31734b2c-9ada-4535-bae1-86683767ee95 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:54.092900 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08bfc4-6197-44ba-90eb-7b93f535abe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:54.092900 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08bfc4-6197-44ba-90eb-7b93f535abe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:54.092900 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08bfc4-6197-44ba-90eb-7b93f535abe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:54.092900 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08bfc4-6197-44ba-90eb-7b93f535abe0 admin admin] PATH_INFO: `/v3/projects/a12b94866d7c448a89bfe0ad9ee874aa/users/910d0584dd6b489e94f3bdeb81272d35/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:54.114053 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08bfc4-6197-44ba-90eb-7b93f535abe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a12b94866d7c448a89bfe0ad9ee874aa, user_id=910d0584dd6b489e94f3bdeb81272d35, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:54.122291 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08bfc4-6197-44ba-90eb-7b93f535abe0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:54.152668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1014/2033] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:54 2022] PUT /identity/v3/projects/a12b94866d7c448a89bfe0ad9ee874aa/users/910d0584dd6b489e94f3bdeb81272d35/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:54.171507 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c70e6-46db-4ac4-b569-82b113223b6f None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:54.171841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c70e6-46db-4ac4-b569-82b113223b6f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:54.172149 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c70e6-46db-4ac4-b569-82b113223b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:54.218647 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-ed8c70e6-46db-4ac4-b569-82b113223b6f None None] MFA Rules not processed for user `910d0584dd6b489e94f3bdeb81272d35`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:54.220584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ed8c70e6-46db-4ac4-b569-82b113223b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:54.339738 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1020/2034] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:54.357484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08bfc4-6197-44ba-90eb-7b93f535abe0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:54.359403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3b08bfc4-6197-44ba-90eb-7b93f535abe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:54.428874 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:54.444874 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08bfc4-6197-44ba-90eb-7b93f535abe0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:54.449352 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fb756-6886-4c1f-93fe-cf5f3672ec03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:54.449788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fb756-6886-4c1f-93fe-cf5f3672ec03 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:54.450230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fb756-6886-4c1f-93fe-cf5f3672ec03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:54.450597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fb756-6886-4c1f-93fe-cf5f3672ec03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:54.465387 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fb756-6886-4c1f-93fe-cf5f3672ec03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:54.473252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fb756-6886-4c1f-93fe-cf5f3672ec03 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:54.501983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1015/2035] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:54.517200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c70e6-46db-4ac4-b569-82b113223b6f None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:54.649889 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c70e6-46db-4ac4-b569-82b113223b6f None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:54.655333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5263af-4c64-4e4e-af84-e6c826c1df27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:54.656057 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5263af-4c64-4e4e-af84-e6c826c1df27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:54.656336 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5263af-4c64-4e4e-af84-e6c826c1df27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:54.656611 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5263af-4c64-4e4e-af84-e6c826c1df27 admin admin] PATH_INFO: `/v3/users/910d0584dd6b489e94f3bdeb81272d35` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:54.657871 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5263af-4c64-4e4e-af84-e6c826c1df27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=910d0584dd6b489e94f3bdeb81272d35)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:54.661709 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5263af-4c64-4e4e-af84-e6c826c1df27 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:54.721663 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:54.747636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ea5263af-4c64-4e4e-af84-e6c826c1df27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '910d0584dd6b489e94f3bdeb81272d35'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:54.748974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ea5263af-4c64-4e4e-af84-e6c826c1df27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '910d0584dd6b489e94f3bdeb81272d35'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:54.768198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ea5263af-4c64-4e4e-af84-e6c826c1df27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '910d0584dd6b489e94f3bdeb81272d35'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:54.778277 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ea5263af-4c64-4e4e-af84-e6c826c1df27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '910d0584dd6b489e94f3bdeb81272d35'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:54.779423 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ea5263af-4c64-4e4e-af84-e6c826c1df27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '910d0584dd6b489e94f3bdeb81272d35'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:54.812760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1021/2036] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:05:54 2022] DELETE /identity/v3/users/910d0584dd6b489e94f3bdeb81272d35 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:54.817665 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d94b02-f105-4d56-91d6-baf8c572aa5a None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:54.817665 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d94b02-f105-4d56-91d6-baf8c572aa5a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:54.817873 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d94b02-f105-4d56-91d6-baf8c572aa5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:54.842589 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d2d94b02-f105-4d56-91d6-baf8c572aa5a None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:54.844279 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d2d94b02-f105-4d56-91d6-baf8c572aa5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:54.960026 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1016/2037] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:54.966566 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5263af-4c64-4e4e-af84-e6c826c1df27 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:54.968150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ea5263af-4c64-4e4e-af84-e6c826c1df27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:55.032750 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5263af-4c64-4e4e-af84-e6c826c1df27 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:55.037236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eccad5-b657-4d86-bba1-430cace60058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:55.038162 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eccad5-b657-4d86-bba1-430cace60058 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:55.038510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eccad5-b657-4d86-bba1-430cace60058 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:55.038879 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eccad5-b657-4d86-bba1-430cace60058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:55.054274 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eccad5-b657-4d86-bba1-430cace60058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:55.054274 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eccad5-b657-4d86-bba1-430cace60058 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:55.061926 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1022/2038] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:05:55.374005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d94b02-f105-4d56-91d6-baf8c572aa5a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:55.375637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d2d94b02-f105-4d56-91d6-baf8c572aa5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:55.392080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d94b02-f105-4d56-91d6-baf8c572aa5a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:55.397549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7017991b-0c5f-419e-abdf-52ea4b90df27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:55.398304 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017991b-0c5f-419e-abdf-52ea4b90df27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:55.398662 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017991b-0c5f-419e-abdf-52ea4b90df27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:55.398951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017991b-0c5f-419e-abdf-52ea4b90df27 admin admin] PATH_INFO: `/v3/projects/a12b94866d7c448a89bfe0ad9ee874aa` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:55.400176 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7017991b-0c5f-419e-abdf-52ea4b90df27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a12b94866d7c448a89bfe0ad9ee874aa)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:55.402427 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7017991b-0c5f-419e-abdf-52ea4b90df27 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:55.491628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7017991b-0c5f-419e-abdf-52ea4b90df27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a12b94866d7c448a89bfe0ad9ee874aa', 'request_id': 'req-7017991b-0c5f-419e-abdf-52ea4b90df27'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:55.510223 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7017991b-0c5f-419e-abdf-52ea4b90df27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a12b94866d7c448a89bfe0ad9ee874aa', 'request_id': 'req-7017991b-0c5f-419e-abdf-52ea4b90df27'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:55.516995 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7017991b-0c5f-419e-abdf-52ea4b90df27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a12b94866d7c448a89bfe0ad9ee874aa', 'request_id': 'req-7017991b-0c5f-419e-abdf-52ea4b90df27'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:55.529839 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7017991b-0c5f-419e-abdf-52ea4b90df27 admin admin] The token cache is being invalidate because project a12b94866d7c448a89bfe0ad9ee874aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:05:55.530182 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7017991b-0c5f-419e-abdf-52ea4b90df27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:55.531269 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7017991b-0c5f-419e-abdf-52ea4b90df27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:05:55.533120 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1017/2039] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:05:55 2022] DELETE /identity/v3/projects/a12b94866d7c448a89bfe0ad9ee874aa => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:55.539335 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206a8b8-cef4-42a7-b71e-fc68dd3d972f None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:55.540302 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206a8b8-cef4-42a7-b71e-fc68dd3d972f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:55.540756 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206a8b8-cef4-42a7-b71e-fc68dd3d972f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:55.564657 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-6206a8b8-cef4-42a7-b71e-fc68dd3d972f None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:55.565816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6206a8b8-cef4-42a7-b71e-fc68dd3d972f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:55.688398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1023/2040] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:05:55.695981 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbba1aa-42f9-459f-9af8-eef8d09186ba None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:55.696277 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbba1aa-42f9-459f-9af8-eef8d09186ba None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:55.696555 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbba1aa-42f9-459f-9af8-eef8d09186ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:55.732059 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-afbba1aa-42f9-459f-9af8-eef8d09186ba None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:55.734670 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-afbba1aa-42f9-459f-9af8-eef8d09186ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:55.764718 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1018/2041] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:55.769110 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206a8b8-cef4-42a7-b71e-fc68dd3d972f None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:55.781502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206a8b8-cef4-42a7-b71e-fc68dd3d972f None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:55.791707 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611b789-3297-47ff-82e1-f3431daaf0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:55.792381 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611b789-3297-47ff-82e1-f3431daaf0f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:55.792687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611b789-3297-47ff-82e1-f3431daaf0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:55.792970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611b789-3297-47ff-82e1-f3431daaf0f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:55.795994 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9611b789-3297-47ff-82e1-f3431daaf0f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:05:55.796312 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9611b789-3297-47ff-82e1-f3431daaf0f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:55.798016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9611b789-3297-47ff-82e1-f3431daaf0f8 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:55.824817 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1024/2042] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:05:55.827065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbba1aa-42f9-459f-9af8-eef8d09186ba None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:55.845665 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbba1aa-42f9-459f-9af8-eef8d09186ba None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:55.851990 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08685ab-c751-4183-ba26-8eafc2f005ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:55.853584 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08685ab-c751-4183-ba26-8eafc2f005ef admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:55.853809 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08685ab-c751-4183-ba26-8eafc2f005ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:55.854096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08685ab-c751-4183-ba26-8eafc2f005ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:55.854769 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08685ab-c751-4183-ba26-8eafc2f005ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:55.858074 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08685ab-c751-4183-ba26-8eafc2f005ef admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:55.901629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1019/2043] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:05:55 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:55.905340 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611b789-3297-47ff-82e1-f3431daaf0f8 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:55.921038 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611b789-3297-47ff-82e1-f3431daaf0f8 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:55.985995 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:56.018675 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e070e97-861e-4248-965c-f1272f950119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:56.019311 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e070e97-861e-4248-965c-f1272f950119 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:56.019716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e070e97-861e-4248-965c-f1272f950119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:56.020163 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e070e97-861e-4248-965c-f1272f950119 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:56.020904 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e070e97-861e-4248-965c-f1272f950119 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:56.022927 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e070e97-861e-4248-965c-f1272f950119 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:56.085043 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1025/2044] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:05:55 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:05:56.090274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08685ab-c751-4183-ba26-8eafc2f005ef admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:56.103857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08685ab-c751-4183-ba26-8eafc2f005ef admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:56.130393 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7459a-0f24-4678-a63c-a5deed0a1b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:56.131035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7459a-0f24-4678-a63c-a5deed0a1b58 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:56.131322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7459a-0f24-4678-a63c-a5deed0a1b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:56.131632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7459a-0f24-4678-a63c-a5deed0a1b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:56.132909 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7459a-0f24-4678-a63c-a5deed0a1b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:56.135072 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7459a-0f24-4678-a63c-a5deed0a1b58 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:56.151641 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1020/2045] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:05:56.153077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e070e97-861e-4248-965c-f1272f950119 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:56.168535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e070e97-861e-4248-965c-f1272f950119 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:56.174529 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42824b01-8446-428e-a2f6-e5bafa55fea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:56.175158 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42824b01-8446-428e-a2f6-e5bafa55fea7 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:56.175445 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42824b01-8446-428e-a2f6-e5bafa55fea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:56.175772 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42824b01-8446-428e-a2f6-e5bafa55fea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:56.176499 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42824b01-8446-428e-a2f6-e5bafa55fea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:56.178129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42824b01-8446-428e-a2f6-e5bafa55fea7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:56.197563 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1026/2046] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:05:56.201447 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7459a-0f24-4678-a63c-a5deed0a1b58 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:56.216525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7459a-0f24-4678-a63c-a5deed0a1b58 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:56.225900 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cab80-1454-4685-807f-650d388bdba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:56.227072 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cab80-1454-4685-807f-650d388bdba0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:56.227363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cab80-1454-4685-807f-650d388bdba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:56.227695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cab80-1454-4685-807f-650d388bdba0 admin admin] PATH_INFO: `/v3/projects/21e8583c61244c38beca5bc42d669e0f/users/b03519a909074011846fb9a7405ae36e/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:56.253018 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cab80-1454-4685-807f-650d388bdba0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21e8583c61244c38beca5bc42d669e0f, user_id=b03519a909074011846fb9a7405ae36e, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:56.255931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cab80-1454-4685-807f-650d388bdba0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:56.273607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1021/2047] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:05:56 2022] PUT /identity/v3/projects/21e8583c61244c38beca5bc42d669e0f/users/b03519a909074011846fb9a7405ae36e/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:05:56.284392 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a25709-c417-41c1-9b4a-f51efe64e9d5 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:56.284712 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a25709-c417-41c1-9b4a-f51efe64e9d5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:56.284999 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a25709-c417-41c1-9b4a-f51efe64e9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:56.325554 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e3a25709-c417-41c1-9b4a-f51efe64e9d5 None None] MFA Rules not processed for user `b03519a909074011846fb9a7405ae36e`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:05:56.326808 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e3a25709-c417-41c1-9b4a-f51efe64e9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:56.414992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1027/2048] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:05:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:05:56.428032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cab80-1454-4685-807f-650d388bdba0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:05:56.430094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2e2cab80-1454-4685-807f-650d388bdba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:05:56.442125 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:05:56.515057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cab80-1454-4685-807f-650d388bdba0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:05:56.519368 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f0fdc-2d5d-4082-b27f-06b7f2e77ca9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:05:56.520280 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f0fdc-2d5d-4082-b27f-06b7f2e77ca9 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:05:56.521673 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f0fdc-2d5d-4082-b27f-06b7f2e77ca9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:05:56.521673 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f0fdc-2d5d-4082-b27f-06b7f2e77ca9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:05:56.533718 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f0fdc-2d5d-4082-b27f-06b7f2e77ca9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:05:56.535434 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f0fdc-2d5d-4082-b27f-06b7f2e77ca9 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:05:56.545441 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1022/2049] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:05:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:06:06.411770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:06:06.540792 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=23 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:06:08.173700 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a25709-c417-41c1-9b4a-f51efe64e9d5 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:08.236645 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a25709-c417-41c1-9b4a-f51efe64e9d5 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:08.241249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379386bb-f39b-480a-a8ad-b72d65a6aa3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:08.241897 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379386bb-f39b-480a-a8ad-b72d65a6aa3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:08.242176 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379386bb-f39b-480a-a8ad-b72d65a6aa3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:08.242452 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379386bb-f39b-480a-a8ad-b72d65a6aa3e admin admin] PATH_INFO: `/v3/users/b03519a909074011846fb9a7405ae36e` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:08.243582 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379386bb-f39b-480a-a8ad-b72d65a6aa3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b03519a909074011846fb9a7405ae36e)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:08.245508 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379386bb-f39b-480a-a8ad-b72d65a6aa3e admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:08.295090 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-379386bb-f39b-480a-a8ad-b72d65a6aa3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b03519a909074011846fb9a7405ae36e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:08.295779 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-379386bb-f39b-480a-a8ad-b72d65a6aa3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b03519a909074011846fb9a7405ae36e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:08.303151 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-379386bb-f39b-480a-a8ad-b72d65a6aa3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b03519a909074011846fb9a7405ae36e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:08.310047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-379386bb-f39b-480a-a8ad-b72d65a6aa3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b03519a909074011846fb9a7405ae36e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:08.310687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-379386bb-f39b-480a-a8ad-b72d65a6aa3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b03519a909074011846fb9a7405ae36e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:08.328181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1028/2050] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:06:08 2022] DELETE /identity/v3/users/b03519a909074011846fb9a7405ae36e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:06:08.333193 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66503ac4-9e30-4176-9f91-a6d4c302c9ca None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:08.333699 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66503ac4-9e30-4176-9f91-a6d4c302c9ca None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:08.333825 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66503ac4-9e30-4176-9f91-a6d4c302c9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:08.352974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-66503ac4-9e30-4176-9f91-a6d4c302c9ca None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:06:08.354149 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-66503ac4-9e30-4176-9f91-a6d4c302c9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:08.411297 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1023/2051] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:06:08.420976 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379386bb-f39b-480a-a8ad-b72d65a6aa3e admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:08.422367 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-379386bb-f39b-480a-a8ad-b72d65a6aa3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:08.485066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379386bb-f39b-480a-a8ad-b72d65a6aa3e admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:08.488769 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ea5d0-f014-406e-94fe-126de097ef40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:08.489373 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ea5d0-f014-406e-94fe-126de097ef40 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:08.489654 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ea5d0-f014-406e-94fe-126de097ef40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:08.489928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ea5d0-f014-406e-94fe-126de097ef40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:08.499162 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ea5d0-f014-406e-94fe-126de097ef40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:08.500649 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ea5d0-f014-406e-94fe-126de097ef40 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:08.510829 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1029/2052] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:06:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:06:08.786756 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66503ac4-9e30-4176-9f91-a6d4c302c9ca None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:08.788199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-66503ac4-9e30-4176-9f91-a6d4c302c9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:08.809848 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66503ac4-9e30-4176-9f91-a6d4c302c9ca None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:08.814730 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4e850-1e0c-4b5e-bdb8-6d4e88ac1106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:08.815342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4e850-1e0c-4b5e-bdb8-6d4e88ac1106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:08.815539 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4e850-1e0c-4b5e-bdb8-6d4e88ac1106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:08.815853 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4e850-1e0c-4b5e-bdb8-6d4e88ac1106 admin admin] PATH_INFO: `/v3/projects/21e8583c61244c38beca5bc42d669e0f` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:08.817571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4e850-1e0c-4b5e-bdb8-6d4e88ac1106 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21e8583c61244c38beca5bc42d669e0f)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:08.819495 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4e850-1e0c-4b5e-bdb8-6d4e88ac1106 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:08.879265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-69c4e850-1e0c-4b5e-bdb8-6d4e88ac1106 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21e8583c61244c38beca5bc42d669e0f', 'request_id': 'req-69c4e850-1e0c-4b5e-bdb8-6d4e88ac1106'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:08.886977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-69c4e850-1e0c-4b5e-bdb8-6d4e88ac1106 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21e8583c61244c38beca5bc42d669e0f', 'request_id': 'req-69c4e850-1e0c-4b5e-bdb8-6d4e88ac1106'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:08.893467 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-69c4e850-1e0c-4b5e-bdb8-6d4e88ac1106 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21e8583c61244c38beca5bc42d669e0f', 'request_id': 'req-69c4e850-1e0c-4b5e-bdb8-6d4e88ac1106'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:08.905393 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-69c4e850-1e0c-4b5e-bdb8-6d4e88ac1106 admin admin] The token cache is being invalidate because project 21e8583c61244c38beca5bc42d669e0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:06:08.905681 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-69c4e850-1e0c-4b5e-bdb8-6d4e88ac1106 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:08.906332 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-69c4e850-1e0c-4b5e-bdb8-6d4e88ac1106 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:08.907904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1024/2053] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:06:08 2022] DELETE /identity/v3/projects/21e8583c61244c38beca5bc42d669e0f => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:06:18.506416 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:06:18.902197 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:06:27.589240 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96051cb-0fb0-4aed-866b-ef0410c10ba3 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:27.589791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96051cb-0fb0-4aed-866b-ef0410c10ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:27.589791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96051cb-0fb0-4aed-866b-ef0410c10ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:27.614137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d96051cb-0fb0-4aed-866b-ef0410c10ba3 None None] MFA Rules not processed for user `9bca70e61b80499eb07a668e11534257`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:06:27.615372 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d96051cb-0fb0-4aed-866b-ef0410c10ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:27.716560 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1030/2054] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:06:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:06:37.703381 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=25 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:06:38.304660 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a8883-9a11-4084-90e2-7e4d90738723 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:38.305153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a8883-9a11-4084-90e2-7e4d90738723 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:38.305371 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a8883-9a11-4084-90e2-7e4d90738723 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:38.306856 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1025/2055] 10.222.0.42 () {54 vars in 921 bytes} [Wed Sep 14 01:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:06:38.319669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e729e6-acfc-4ac0-b8a2-182ad807c577 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:38.320143 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e729e6-acfc-4ac0-b8a2-182ad807c577 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:38.320585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e729e6-acfc-4ac0-b8a2-182ad807c577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:38.340333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-67e729e6-acfc-4ac0-b8a2-182ad807c577 None None] MFA Rules not processed for user `18798838de284c84b5b838b3530a2c43`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:06:38.341543 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-67e729e6-acfc-4ac0-b8a2-182ad807c577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:38.420800 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1031/2056] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Sep 14 01:06:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:06:38.435142 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a8883-9a11-4084-90e2-7e4d90738723 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:38.437110 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1b6a8883-9a11-4084-90e2-7e4d90738723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:38.456432 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a8883-9a11-4084-90e2-7e4d90738723 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:38.462426 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e729e6-acfc-4ac0-b8a2-182ad807c577 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:38.464167 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-67e729e6-acfc-4ac0-b8a2-182ad807c577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:38.466785 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7249d-b458-4460-96bf-0f651f519aba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18798838de284c84b5b838b3530a2c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:38.467420 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7249d-b458-4460-96bf-0f651f519aba service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:38.467886 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7249d-b458-4460-96bf-0f651f519aba service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:38.468339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7249d-b458-4460-96bf-0f651f519aba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:38.481983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a7249d-b458-4460-96bf-0f651f519aba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:38.484779 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a7249d-b458-4460-96bf-0f651f519aba service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:38.499420 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1026/2057] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:06:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:06:38.544467 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e729e6-acfc-4ac0-b8a2-182ad807c577 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:38.549769 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4390c34-0e3d-41f1-a9ab-8bea9eecaf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:38.550495 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4390c34-0e3d-41f1-a9ab-8bea9eecaf95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:38.550782 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4390c34-0e3d-41f1-a9ab-8bea9eecaf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:38.551059 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4390c34-0e3d-41f1-a9ab-8bea9eecaf95 admin admin] PATH_INFO: `/v3/users/017a9de44877436fa9531beb756c54c2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:38.552244 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4390c34-0e3d-41f1-a9ab-8bea9eecaf95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=017a9de44877436fa9531beb756c54c2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:38.554641 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4390c34-0e3d-41f1-a9ab-8bea9eecaf95 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:38.607656 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e4390c34-0e3d-41f1-a9ab-8bea9eecaf95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '017a9de44877436fa9531beb756c54c2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:38.608337 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e4390c34-0e3d-41f1-a9ab-8bea9eecaf95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '017a9de44877436fa9531beb756c54c2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:38.616409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e4390c34-0e3d-41f1-a9ab-8bea9eecaf95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '017a9de44877436fa9531beb756c54c2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:38.623571 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e4390c34-0e3d-41f1-a9ab-8bea9eecaf95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '017a9de44877436fa9531beb756c54c2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:38.624318 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e4390c34-0e3d-41f1-a9ab-8bea9eecaf95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '017a9de44877436fa9531beb756c54c2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:38.643724 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1032/2058] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:06:38 2022] DELETE /identity/v3/users/017a9de44877436fa9531beb756c54c2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:06:38.829148 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7249d-b458-4460-96bf-0f651f519aba service placement] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:38.853728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7249d-b458-4460-96bf-0f651f519aba service placement] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:38.940415 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff18f3-fd5c-49da-adae-f0a18bff6cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:38.941124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff18f3-fd5c-49da-adae-f0a18bff6cbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:38.941516 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff18f3-fd5c-49da-adae-f0a18bff6cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:38.941964 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff18f3-fd5c-49da-adae-f0a18bff6cbe admin admin] PATH_INFO: `/v3/projects/832072fd5b984b5b91d249c5733452b8` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:38.944450 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff18f3-fd5c-49da-adae-f0a18bff6cbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=832072fd5b984b5b91d249c5733452b8)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:38.946663 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff18f3-fd5c-49da-adae-f0a18bff6cbe admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:39.009450 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eeff18f3-fd5c-49da-adae-f0a18bff6cbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '832072fd5b984b5b91d249c5733452b8', 'request_id': 'req-eeff18f3-fd5c-49da-adae-f0a18bff6cbe'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:39.019136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eeff18f3-fd5c-49da-adae-f0a18bff6cbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '832072fd5b984b5b91d249c5733452b8', 'request_id': 'req-eeff18f3-fd5c-49da-adae-f0a18bff6cbe'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:39.028187 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eeff18f3-fd5c-49da-adae-f0a18bff6cbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '832072fd5b984b5b91d249c5733452b8', 'request_id': 'req-eeff18f3-fd5c-49da-adae-f0a18bff6cbe'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:39.044831 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eeff18f3-fd5c-49da-adae-f0a18bff6cbe admin admin] The token cache is being invalidate because project 832072fd5b984b5b91d249c5733452b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:06:39.045178 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eeff18f3-fd5c-49da-adae-f0a18bff6cbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:39.045838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eeff18f3-fd5c-49da-adae-f0a18bff6cbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:39.051162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1027/2059] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:06:38 2022] DELETE /identity/v3/projects/832072fd5b984b5b91d249c5733452b8 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:06:39.052885 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eef807-415d-4486-9ee2-6bd65604782c None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:39.053371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eef807-415d-4486-9ee2-6bd65604782c None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:39.053471 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eef807-415d-4486-9ee2-6bd65604782c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:39.079596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a0eef807-415d-4486-9ee2-6bd65604782c None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:06:39.081758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a0eef807-415d-4486-9ee2-6bd65604782c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:39.102216 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:06:39.153416 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1033/2060] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:06:39.161434 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66411eeb-337b-4ad4-9b81-7d46a2ac0272 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:39.161924 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66411eeb-337b-4ad4-9b81-7d46a2ac0272 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:39.162303 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66411eeb-337b-4ad4-9b81-7d46a2ac0272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:39.176820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eef807-415d-4486-9ee2-6bd65604782c None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:39.178238 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a0eef807-415d-4486-9ee2-6bd65604782c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:39.182734 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-66411eeb-337b-4ad4-9b81-7d46a2ac0272 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:06:39.184028 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-66411eeb-337b-4ad4-9b81-7d46a2ac0272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:39.191115 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eef807-415d-4486-9ee2-6bd65604782c None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:39.193447 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1028/2061] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:06:39.197664 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66411eeb-337b-4ad4-9b81-7d46a2ac0272 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:39.200647 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebbb03-afaa-42a0-bf3b-3eaed4792160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:39.201314 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebbb03-afaa-42a0-bf3b-3eaed4792160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:39.201595 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebbb03-afaa-42a0-bf3b-3eaed4792160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:39.201943 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebbb03-afaa-42a0-bf3b-3eaed4792160 admin admin] PATH_INFO: `/v3/users/be12ec8762ec414892642843701821ce` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:39.203519 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beebbb03-afaa-42a0-bf3b-3eaed4792160 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be12ec8762ec414892642843701821ce)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:39.204988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beebbb03-afaa-42a0-bf3b-3eaed4792160 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:39.207527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66411eeb-337b-4ad4-9b81-7d46a2ac0272 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:39.212157 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3eef24-777f-428d-9854-d493f5832556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:39.212752 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3eef24-777f-428d-9854-d493f5832556 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:39.213100 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3eef24-777f-428d-9854-d493f5832556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:39.213378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3eef24-777f-428d-9854-d493f5832556 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:39.214026 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3eef24-777f-428d-9854-d493f5832556 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:06:39.214332 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3eef24-777f-428d-9854-d493f5832556 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:39.215954 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3eef24-777f-428d-9854-d493f5832556 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:39.228649 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1029/2062] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:06:39.232950 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3eef24-777f-428d-9854-d493f5832556 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:39.243720 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3eef24-777f-428d-9854-d493f5832556 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:39.248650 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74858ae3-fdea-45d3-a4bf-e1ce86a1cbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:39.249392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74858ae3-fdea-45d3-a4bf-e1ce86a1cbdb admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:39.249680 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74858ae3-fdea-45d3-a4bf-e1ce86a1cbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:39.250010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74858ae3-fdea-45d3-a4bf-e1ce86a1cbdb admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:39.250650 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74858ae3-fdea-45d3-a4bf-e1ce86a1cbdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:39.253322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74858ae3-fdea-45d3-a4bf-e1ce86a1cbdb admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:39.257872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-beebbb03-afaa-42a0-bf3b-3eaed4792160 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be12ec8762ec414892642843701821ce'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:39.258780 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-beebbb03-afaa-42a0-bf3b-3eaed4792160 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be12ec8762ec414892642843701821ce'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:39.266720 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-beebbb03-afaa-42a0-bf3b-3eaed4792160 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be12ec8762ec414892642843701821ce'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:39.275727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-beebbb03-afaa-42a0-bf3b-3eaed4792160 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be12ec8762ec414892642843701821ce'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:39.276689 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-beebbb03-afaa-42a0-bf3b-3eaed4792160 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be12ec8762ec414892642843701821ce'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:39.281217 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1030/2063] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:06:39 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:06:39.286003 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74858ae3-fdea-45d3-a4bf-e1ce86a1cbdb admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:39.287728 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-74858ae3-fdea-45d3-a4bf-e1ce86a1cbdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:39.303210 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1034/2064] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:06:39 2022] DELETE /identity/v3/users/be12ec8762ec414892642843701821ce => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:06:39.308907 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526bf54-b8fd-4800-8431-989e318c44ff None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:39.309200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526bf54-b8fd-4800-8431-989e318c44ff None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:39.309484 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526bf54-b8fd-4800-8431-989e318c44ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:39.329123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:06:39.329690 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-b526bf54-b8fd-4800-8431-989e318c44ff None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:06:39.331332 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b526bf54-b8fd-4800-8431-989e318c44ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:39.363694 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74858ae3-fdea-45d3-a4bf-e1ce86a1cbdb admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:39.395757 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1035/2065] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:06:39.405634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526bf54-b8fd-4800-8431-989e318c44ff None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:39.407104 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b526bf54-b8fd-4800-8431-989e318c44ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:39.424764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba5a13-3b90-4696-b53a-e01aee927f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:39.425426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba5a13-3b90-4696-b53a-e01aee927f55 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:39.425709 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba5a13-3b90-4696-b53a-e01aee927f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:39.425986 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba5a13-3b90-4696-b53a-e01aee927f55 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:39.426619 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba5a13-3b90-4696-b53a-e01aee927f55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:39.428634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba5a13-3b90-4696-b53a-e01aee927f55 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:39.442934 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:06:39.451776 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1031/2066] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:06:39 2022] POST /identity/v3/users => generated 284 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:06:39.457014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba5a13-3b90-4696-b53a-e01aee927f55 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:39.475758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526bf54-b8fd-4800-8431-989e318c44ff None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:39.476741 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba5a13-3b90-4696-b53a-e01aee927f55 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:39.480528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7a752-b69c-4805-932e-a60cb24541a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:39.481264 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7a752-b69c-4805-932e-a60cb24541a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:39.481562 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa727e-df46-4470-8450-1cdb8c503d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:39.481762 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7a752-b69c-4805-932e-a60cb24541a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:39.482147 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7a752-b69c-4805-932e-a60cb24541a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:39.482253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa727e-df46-4470-8450-1cdb8c503d1d admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:39.482439 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa727e-df46-4470-8450-1cdb8c503d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:39.482734 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa727e-df46-4470-8450-1cdb8c503d1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:39.483386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa727e-df46-4470-8450-1cdb8c503d1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:39.485029 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa727e-df46-4470-8450-1cdb8c503d1d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:39.493340 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f7a752-b69c-4805-932e-a60cb24541a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:39.494792 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f7a752-b69c-4805-932e-a60cb24541a3 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:39.496940 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1032/2067] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:06:39.500940 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa727e-df46-4470-8450-1cdb8c503d1d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:39.505458 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1036/2068] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:06:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:06:39.511463 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa727e-df46-4470-8450-1cdb8c503d1d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:39.516596 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed8bdc-1a45-498f-979a-d8addfc50e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:39.517284 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed8bdc-1a45-498f-979a-d8addfc50e81 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:39.517708 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed8bdc-1a45-498f-979a-d8addfc50e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:39.518063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed8bdc-1a45-498f-979a-d8addfc50e81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:39.518852 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed8bdc-1a45-498f-979a-d8addfc50e81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:39.520677 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed8bdc-1a45-498f-979a-d8addfc50e81 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:39.532751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1033/2069] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:06:39.537185 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7a752-b69c-4805-932e-a60cb24541a3 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:39.546957 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7a752-b69c-4805-932e-a60cb24541a3 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:39.552613 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95548ae1-7eb0-4880-af9c-19f98d009ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:39.553096 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95548ae1-7eb0-4880-af9c-19f98d009ba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:39.553379 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95548ae1-7eb0-4880-af9c-19f98d009ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:39.553713 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95548ae1-7eb0-4880-af9c-19f98d009ba2 admin admin] PATH_INFO: `/v3/projects/f79f3054de6849ccbd4ac3f03221bb13/users/79a48d543d2d431b930fcd600537767d/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:39.570506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95548ae1-7eb0-4880-af9c-19f98d009ba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f79f3054de6849ccbd4ac3f03221bb13, user_id=79a48d543d2d431b930fcd600537767d, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:39.573404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95548ae1-7eb0-4880-af9c-19f98d009ba2 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:39.587148 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1037/2070] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:06:39 2022] PUT /identity/v3/projects/f79f3054de6849ccbd4ac3f03221bb13/users/79a48d543d2d431b930fcd600537767d/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:06:39.597499 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127bcca-1f97-4d97-addd-52d31ea60817 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:39.597866 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127bcca-1f97-4d97-addd-52d31ea60817 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:39.598214 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b127bcca-1f97-4d97-addd-52d31ea60817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:39.629693 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b127bcca-1f97-4d97-addd-52d31ea60817 None None] MFA Rules not processed for user `79a48d543d2d431b930fcd600537767d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:06:39.630917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b127bcca-1f97-4d97-addd-52d31ea60817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:39.708005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1034/2071] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:06:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:06:39.718794 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95548ae1-7eb0-4880-af9c-19f98d009ba2 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:39.721203 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-95548ae1-7eb0-4880-af9c-19f98d009ba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:39.783434 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b127bcca-1f97-4d97-addd-52d31ea60817 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:39.785214 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b127bcca-1f97-4d97-addd-52d31ea60817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:39.785836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95548ae1-7eb0-4880-af9c-19f98d009ba2 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:39.790906 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e0dc3-f47b-47fd-935c-9c48fdac8a7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:39.792103 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e0dc3-f47b-47fd-935c-9c48fdac8a7d service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:39.792396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e0dc3-f47b-47fd-935c-9c48fdac8a7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:39.792676 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e0dc3-f47b-47fd-935c-9c48fdac8a7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:39.803151 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e0dc3-f47b-47fd-935c-9c48fdac8a7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:39.804647 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e0dc3-f47b-47fd-935c-9c48fdac8a7d service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:39.819097 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1038/2072] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:06:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:06:39.834205 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:06:39.857252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b127bcca-1f97-4d97-addd-52d31ea60817 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:39.862553 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e483e4-ce8f-44ef-89b0-2402241e025b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:39.863122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e483e4-ce8f-44ef-89b0-2402241e025b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:39.863425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e483e4-ce8f-44ef-89b0-2402241e025b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:39.863734 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e483e4-ce8f-44ef-89b0-2402241e025b admin admin] PATH_INFO: `/v3/projects/38e0fb44599648b99e9d59701bca4a90` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:39.865048 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e483e4-ce8f-44ef-89b0-2402241e025b admin admin] RBAC: Authorizing `identity:delete_project(project_id=38e0fb44599648b99e9d59701bca4a90)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:39.866961 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e483e4-ce8f-44ef-89b0-2402241e025b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:39.916168 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f5e483e4-ce8f-44ef-89b0-2402241e025b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38e0fb44599648b99e9d59701bca4a90', 'request_id': 'req-f5e483e4-ce8f-44ef-89b0-2402241e025b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:39.925295 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f5e483e4-ce8f-44ef-89b0-2402241e025b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38e0fb44599648b99e9d59701bca4a90', 'request_id': 'req-f5e483e4-ce8f-44ef-89b0-2402241e025b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:39.931755 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f5e483e4-ce8f-44ef-89b0-2402241e025b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38e0fb44599648b99e9d59701bca4a90', 'request_id': 'req-f5e483e4-ce8f-44ef-89b0-2402241e025b'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:39.944440 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f5e483e4-ce8f-44ef-89b0-2402241e025b admin admin] The token cache is being invalidate because project 38e0fb44599648b99e9d59701bca4a90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:06:39.944730 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f5e483e4-ce8f-44ef-89b0-2402241e025b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:39.945423 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f5e483e4-ce8f-44ef-89b0-2402241e025b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:39.946935 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1035/2073] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:06:39 2022] DELETE /identity/v3/projects/38e0fb44599648b99e9d59701bca4a90 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:06:39.952089 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b04907-6347-444e-9366-dfa773dc6c9d None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:39.952968 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b04907-6347-444e-9366-dfa773dc6c9d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:39.953267 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b04907-6347-444e-9366-dfa773dc6c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:39.975592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-99b04907-6347-444e-9366-dfa773dc6c9d None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:06:39.976791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-99b04907-6347-444e-9366-dfa773dc6c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:40.054277 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1039/2074] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:06:40.057763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0395b7-2648-4794-8b33-cbf02055ec65 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:40.057763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0395b7-2648-4794-8b33-cbf02055ec65 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:40.057763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0395b7-2648-4794-8b33-cbf02055ec65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:40.097167 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-1e0395b7-2648-4794-8b33-cbf02055ec65 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:06:40.098547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1e0395b7-2648-4794-8b33-cbf02055ec65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:40.107811 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1036/2075] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:06:40.112227 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b04907-6347-444e-9366-dfa773dc6c9d None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:40.145136 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b04907-6347-444e-9366-dfa773dc6c9d None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:40.164053 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbdd56-23fe-4bc5-9e39-5882b2ed0f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:40.164716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbdd56-23fe-4bc5-9e39-5882b2ed0f81 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:40.165004 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbdd56-23fe-4bc5-9e39-5882b2ed0f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:40.165286 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbdd56-23fe-4bc5-9e39-5882b2ed0f81 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:40.165967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbdd56-23fe-4bc5-9e39-5882b2ed0f81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:06:40.166279 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbdd56-23fe-4bc5-9e39-5882b2ed0f81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:40.167941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbdd56-23fe-4bc5-9e39-5882b2ed0f81 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:40.181424 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1040/2076] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:06:40.186094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0395b7-2648-4794-8b33-cbf02055ec65 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:40.198952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0395b7-2648-4794-8b33-cbf02055ec65 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:40.204489 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2fbce-436c-4a4e-aad3-62ee2236fefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:40.205840 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2fbce-436c-4a4e-aad3-62ee2236fefb admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:40.206344 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2fbce-436c-4a4e-aad3-62ee2236fefb admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:40.206903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2fbce-436c-4a4e-aad3-62ee2236fefb admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:40.208004 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f2fbce-436c-4a4e-aad3-62ee2236fefb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:40.209665 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f2fbce-436c-4a4e-aad3-62ee2236fefb admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:40.240546 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1037/2077] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:06:40 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:06:40.244757 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbdd56-23fe-4bc5-9e39-5882b2ed0f81 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:40.256384 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbdd56-23fe-4bc5-9e39-5882b2ed0f81 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:40.292985 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:06:40.318430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1d437-cb06-4bb3-9c2a-35c0bd81870e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:40.319117 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1d437-cb06-4bb3-9c2a-35c0bd81870e admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:40.319402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1d437-cb06-4bb3-9c2a-35c0bd81870e admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:40.319757 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1d437-cb06-4bb3-9c2a-35c0bd81870e admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:40.321016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1d437-cb06-4bb3-9c2a-35c0bd81870e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:40.322940 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1d437-cb06-4bb3-9c2a-35c0bd81870e admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:40.344218 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1041/2078] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:06:40 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:06:40.348428 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2fbce-436c-4a4e-aad3-62ee2236fefb admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:40.358732 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2fbce-436c-4a4e-aad3-62ee2236fefb admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:40.364022 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa25cf-e7cd-4f65-8202-301e707a7864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:40.364621 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa25cf-e7cd-4f65-8202-301e707a7864 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:40.364899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa25cf-e7cd-4f65-8202-301e707a7864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:40.365178 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa25cf-e7cd-4f65-8202-301e707a7864 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:40.365851 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fa25cf-e7cd-4f65-8202-301e707a7864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:40.367487 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fa25cf-e7cd-4f65-8202-301e707a7864 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:40.378561 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1038/2079] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:06:40.384864 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1d437-cb06-4bb3-9c2a-35c0bd81870e admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:40.396148 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1d437-cb06-4bb3-9c2a-35c0bd81870e admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:40.400705 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb58244-718b-4247-b56b-69a1b2c972ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:40.401294 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb58244-718b-4247-b56b-69a1b2c972ed admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:40.401574 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb58244-718b-4247-b56b-69a1b2c972ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:40.401847 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb58244-718b-4247-b56b-69a1b2c972ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:40.402481 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb58244-718b-4247-b56b-69a1b2c972ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:40.404116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb58244-718b-4247-b56b-69a1b2c972ed admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:40.415098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1042/2080] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:06:40.419592 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa25cf-e7cd-4f65-8202-301e707a7864 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:40.430382 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa25cf-e7cd-4f65-8202-301e707a7864 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:40.435069 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a735e-72d4-42fd-9f7e-deb7fe0fb8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:40.435809 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a735e-72d4-42fd-9f7e-deb7fe0fb8b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:40.436093 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a735e-72d4-42fd-9f7e-deb7fe0fb8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:40.436367 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a735e-72d4-42fd-9f7e-deb7fe0fb8b2 admin admin] PATH_INFO: `/v3/projects/0f8eb4de1820448eb1064b0fbb45a259/users/0b9c17d96f3d44589cee145c8ec2e97d/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:40.451352 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a735e-72d4-42fd-9f7e-deb7fe0fb8b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f8eb4de1820448eb1064b0fbb45a259, user_id=0b9c17d96f3d44589cee145c8ec2e97d, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:40.454072 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a735e-72d4-42fd-9f7e-deb7fe0fb8b2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:40.466720 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1039/2081] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:06:40 2022] PUT /identity/v3/projects/0f8eb4de1820448eb1064b0fbb45a259/users/0b9c17d96f3d44589cee145c8ec2e97d/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:06:40.472318 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3c2f2-5171-4bbe-bbb5-a6253c0334d2 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:40.472603 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3c2f2-5171-4bbe-bbb5-a6253c0334d2 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:40.472912 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3c2f2-5171-4bbe-bbb5-a6253c0334d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:40.492411 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-fda3c2f2-5171-4bbe-bbb5-a6253c0334d2 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:06:40.493621 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-fda3c2f2-5171-4bbe-bbb5-a6253c0334d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:40.553186 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1043/2082] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:06:40.562801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a735e-72d4-42fd-9f7e-deb7fe0fb8b2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:40.564623 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-856a735e-72d4-42fd-9f7e-deb7fe0fb8b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:40.624130 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a735e-72d4-42fd-9f7e-deb7fe0fb8b2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:40.628136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62dcbb-9d87-41c9-9c3c-7af97dceedb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:40.628733 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62dcbb-9d87-41c9-9c3c-7af97dceedb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:40.629008 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62dcbb-9d87-41c9-9c3c-7af97dceedb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:40.629278 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62dcbb-9d87-41c9-9c3c-7af97dceedb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:40.638436 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa62dcbb-9d87-41c9-9c3c-7af97dceedb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:40.640407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa62dcbb-9d87-41c9-9c3c-7af97dceedb2 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:40.649877 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1040/2083] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:06:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:06:41.683723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1a894-f9a3-4c72-a652-3256c670cf07 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:41.684241 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1a894-f9a3-4c72-a652-3256c670cf07 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:41.684980 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1a894-f9a3-4c72-a652-3256c670cf07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:41.715108 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-caa1a894-f9a3-4c72-a652-3256c670cf07 None None] MFA Rules not processed for user `0b9c17d96f3d44589cee145c8ec2e97d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:06:41.716383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-caa1a894-f9a3-4c72-a652-3256c670cf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:41.787057 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1044/2084] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:06:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:06:41.798307 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62dcbb-9d87-41c9-9c3c-7af97dceedb2 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:41.800261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-aa62dcbb-9d87-41c9-9c3c-7af97dceedb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:41.811219 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:06:41.872146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62dcbb-9d87-41c9-9c3c-7af97dceedb2 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:41.875857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a4cc4-54eb-43c0-99b8-8dfd71ff483d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:41.876486 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a4cc4-54eb-43c0-99b8-8dfd71ff483d service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:41.876767 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a4cc4-54eb-43c0-99b8-8dfd71ff483d service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:41.877215 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a4cc4-54eb-43c0-99b8-8dfd71ff483d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:41.886661 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a4cc4-54eb-43c0-99b8-8dfd71ff483d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:41.888177 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a4cc4-54eb-43c0-99b8-8dfd71ff483d service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:41.899888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1041/2085] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:06:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:06:44.322257 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e527d2b-0bfc-432b-8671-61d8b203d5eb None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:44.322758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e527d2b-0bfc-432b-8671-61d8b203d5eb None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:44.322758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e527d2b-0bfc-432b-8671-61d8b203d5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:44.344391 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-4e527d2b-0bfc-432b-8671-61d8b203d5eb None None] MFA Rules not processed for user `0b9c17d96f3d44589cee145c8ec2e97d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:06:44.346152 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4e527d2b-0bfc-432b-8671-61d8b203d5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:44.353467 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1045/2086] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:06:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:06:48.721861 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a4cc4-54eb-43c0-99b8-8dfd71ff483d service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:48.723494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-499a4cc4-54eb-43c0-99b8-8dfd71ff483d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:48.739168 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a4cc4-54eb-43c0-99b8-8dfd71ff483d service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:48.748113 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b011e0-2b4e-4fd6-b17e-251430d28bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:48.748895 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b011e0-2b4e-4fd6-b17e-251430d28bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:48.749227 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b011e0-2b4e-4fd6-b17e-251430d28bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:48.749540 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b011e0-2b4e-4fd6-b17e-251430d28bba admin admin] PATH_INFO: `/v3/users/79a48d543d2d431b930fcd600537767d` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:48.751036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b011e0-2b4e-4fd6-b17e-251430d28bba admin admin] RBAC: Authorizing `identity:delete_user(user_id=79a48d543d2d431b930fcd600537767d)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:48.754114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b011e0-2b4e-4fd6-b17e-251430d28bba admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:48.817194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-78b011e0-2b4e-4fd6-b17e-251430d28bba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79a48d543d2d431b930fcd600537767d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:48.818200 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-78b011e0-2b4e-4fd6-b17e-251430d28bba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79a48d543d2d431b930fcd600537767d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:48.828248 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-78b011e0-2b4e-4fd6-b17e-251430d28bba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79a48d543d2d431b930fcd600537767d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:48.829111 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-78b011e0-2b4e-4fd6-b17e-251430d28bba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79a48d543d2d431b930fcd600537767d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:48.837638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-78b011e0-2b4e-4fd6-b17e-251430d28bba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79a48d543d2d431b930fcd600537767d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:48.859374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1042/2087] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:06:48 2022] DELETE /identity/v3/users/79a48d543d2d431b930fcd600537767d => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:06:48.864819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e3f25-44fa-4e57-8bea-fd6ce4f5185f None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:48.865420 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e3f25-44fa-4e57-8bea-fd6ce4f5185f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:48.865610 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e3f25-44fa-4e57-8bea-fd6ce4f5185f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:48.888462 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-0d0e3f25-44fa-4e57-8bea-fd6ce4f5185f None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:06:48.890157 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0d0e3f25-44fa-4e57-8bea-fd6ce4f5185f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:48.927599 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:06:48.958179 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1046/2088] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:06:48.967230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b011e0-2b4e-4fd6-b17e-251430d28bba admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:48.968665 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-78b011e0-2b4e-4fd6-b17e-251430d28bba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:49.027160 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b011e0-2b4e-4fd6-b17e-251430d28bba admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:49.030912 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69007a-c7d1-4d3e-9c89-184bc1816179 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:49.031493 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69007a-c7d1-4d3e-9c89-184bc1816179 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:49.031805 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69007a-c7d1-4d3e-9c89-184bc1816179 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:49.032083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69007a-c7d1-4d3e-9c89-184bc1816179 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:49.043858 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69007a-c7d1-4d3e-9c89-184bc1816179 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:49.045534 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69007a-c7d1-4d3e-9c89-184bc1816179 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:49.056666 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1043/2089] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:06:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:06:49.405727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e3f25-44fa-4e57-8bea-fd6ce4f5185f None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:49.407441 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0d0e3f25-44fa-4e57-8bea-fd6ce4f5185f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:49.422465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e3f25-44fa-4e57-8bea-fd6ce4f5185f None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:49.431012 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02898070-99ff-4c1b-9104-bc99946dfb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:49.431736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02898070-99ff-4c1b-9104-bc99946dfb3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:49.432040 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02898070-99ff-4c1b-9104-bc99946dfb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:49.432261 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02898070-99ff-4c1b-9104-bc99946dfb3a admin admin] PATH_INFO: `/v3/projects/f79f3054de6849ccbd4ac3f03221bb13` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:49.433807 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02898070-99ff-4c1b-9104-bc99946dfb3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f79f3054de6849ccbd4ac3f03221bb13)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:49.436920 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02898070-99ff-4c1b-9104-bc99946dfb3a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:49.503871 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-02898070-99ff-4c1b-9104-bc99946dfb3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f79f3054de6849ccbd4ac3f03221bb13', 'request_id': 'req-02898070-99ff-4c1b-9104-bc99946dfb3a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:49.511322 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-02898070-99ff-4c1b-9104-bc99946dfb3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f79f3054de6849ccbd4ac3f03221bb13', 'request_id': 'req-02898070-99ff-4c1b-9104-bc99946dfb3a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:49.517131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-02898070-99ff-4c1b-9104-bc99946dfb3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f79f3054de6849ccbd4ac3f03221bb13', 'request_id': 'req-02898070-99ff-4c1b-9104-bc99946dfb3a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:49.529004 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-02898070-99ff-4c1b-9104-bc99946dfb3a admin admin] The token cache is being invalidate because project f79f3054de6849ccbd4ac3f03221bb13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:06:49.529365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-02898070-99ff-4c1b-9104-bc99946dfb3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:49.530010 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-02898070-99ff-4c1b-9104-bc99946dfb3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:49.531496 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1047/2090] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:06:49 2022] DELETE /identity/v3/projects/f79f3054de6849ccbd4ac3f03221bb13 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:06:49.536903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017fe9c-dea2-42ea-8203-5c07517872b1 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:49.537267 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017fe9c-dea2-42ea-8203-5c07517872b1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:49.537571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017fe9c-dea2-42ea-8203-5c07517872b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:49.560046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4017fe9c-dea2-42ea-8203-5c07517872b1 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:06:49.561284 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4017fe9c-dea2-42ea-8203-5c07517872b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:49.582166 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:06:49.622986 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1044/2091] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:06:49.632956 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa779159-06f9-42ec-ad5b-cc8c81050bc1 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:49.633273 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa779159-06f9-42ec-ad5b-cc8c81050bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:49.633556 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa779159-06f9-42ec-ad5b-cc8c81050bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:49.655857 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-fa779159-06f9-42ec-ad5b-cc8c81050bc1 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:06:49.657003 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-fa779159-06f9-42ec-ad5b-cc8c81050bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:49.665158 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1048/2092] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:06:49.669502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017fe9c-dea2-42ea-8203-5c07517872b1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:49.679661 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017fe9c-dea2-42ea-8203-5c07517872b1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:49.684580 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898876d-09cf-425e-ab2d-3f33d29f974f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:49.685370 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898876d-09cf-425e-ab2d-3f33d29f974f admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:49.685682 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898876d-09cf-425e-ab2d-3f33d29f974f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:49.685964 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898876d-09cf-425e-ab2d-3f33d29f974f admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:49.686631 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7898876d-09cf-425e-ab2d-3f33d29f974f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:06:49.686946 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7898876d-09cf-425e-ab2d-3f33d29f974f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:49.688637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7898876d-09cf-425e-ab2d-3f33d29f974f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:49.707694 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1045/2093] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:06:49.709034 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa779159-06f9-42ec-ad5b-cc8c81050bc1 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:49.719842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa779159-06f9-42ec-ad5b-cc8c81050bc1 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:49.725198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb53757-e865-441e-833a-883021249a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:49.725817 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb53757-e865-441e-833a-883021249a63 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:49.726103 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb53757-e865-441e-833a-883021249a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:49.726375 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb53757-e865-441e-833a-883021249a63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:49.727057 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb53757-e865-441e-833a-883021249a63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:49.729032 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb53757-e865-441e-833a-883021249a63 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:49.765094 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1049/2094] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:06:49 2022] POST /identity/v3/projects => generated 401 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:06:49.769606 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898876d-09cf-425e-ab2d-3f33d29f974f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:49.780421 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898876d-09cf-425e-ab2d-3f33d29f974f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:49.840249 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c9488-b5c8-41bd-80c7-e573d3ac708e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:49.840854 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c9488-b5c8-41bd-80c7-e573d3ac708e admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:49.841138 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c9488-b5c8-41bd-80c7-e573d3ac708e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:49.841413 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c9488-b5c8-41bd-80c7-e573d3ac708e admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:49.842053 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c9488-b5c8-41bd-80c7-e573d3ac708e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:49.844028 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887c9488-b5c8-41bd-80c7-e573d3ac708e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:49.867378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1046/2095] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:06:49 2022] POST /identity/v3/users => generated 301 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:06:49.871594 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb53757-e865-441e-833a-883021249a63 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:49.883720 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb53757-e865-441e-833a-883021249a63 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:49.889548 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a22e5f-a701-4fc4-b244-905ef3b3dc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:49.890333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a22e5f-a701-4fc4-b244-905ef3b3dc97 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:49.890616 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a22e5f-a701-4fc4-b244-905ef3b3dc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:49.890985 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a22e5f-a701-4fc4-b244-905ef3b3dc97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:49.891722 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a22e5f-a701-4fc4-b244-905ef3b3dc97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:49.894057 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a22e5f-a701-4fc4-b244-905ef3b3dc97 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:49.904842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1050/2096] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:06:49.909469 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c9488-b5c8-41bd-80c7-e573d3ac708e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:49.920059 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c9488-b5c8-41bd-80c7-e573d3ac708e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:49.925750 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17efe1-eb57-47c1-a643-56dfcbf47a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:49.926369 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17efe1-eb57-47c1-a643-56dfcbf47a08 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:49.926653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17efe1-eb57-47c1-a643-56dfcbf47a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:49.926928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17efe1-eb57-47c1-a643-56dfcbf47a08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:49.927605 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17efe1-eb57-47c1-a643-56dfcbf47a08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:49.929219 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17efe1-eb57-47c1-a643-56dfcbf47a08 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:49.944738 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1047/2097] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:06:49.946220 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a22e5f-a701-4fc4-b244-905ef3b3dc97 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:49.957379 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a22e5f-a701-4fc4-b244-905ef3b3dc97 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:49.962454 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d510116-99b2-4c83-83f7-25ea99348dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:49.963198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d510116-99b2-4c83-83f7-25ea99348dca admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:49.963506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d510116-99b2-4c83-83f7-25ea99348dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:49.964421 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d510116-99b2-4c83-83f7-25ea99348dca admin admin] PATH_INFO: `/v3/projects/fb2f1550baa24e66a643c060fa34f8ed/users/375bce18b1f54a6b87978a2574f87b51/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:49.980622 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d510116-99b2-4c83-83f7-25ea99348dca admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb2f1550baa24e66a643c060fa34f8ed, user_id=375bce18b1f54a6b87978a2574f87b51, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:49.983298 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d510116-99b2-4c83-83f7-25ea99348dca admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:49.997263 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1051/2098] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:06:49 2022] PUT /identity/v3/projects/fb2f1550baa24e66a643c060fa34f8ed/users/375bce18b1f54a6b87978a2574f87b51/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:06:50.003090 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28908350-64c9-40f8-9fc7-8d905e63cf30 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:50.003385 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28908350-64c9-40f8-9fc7-8d905e63cf30 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:50.003698 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28908350-64c9-40f8-9fc7-8d905e63cf30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:50.023602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-28908350-64c9-40f8-9fc7-8d905e63cf30 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:06:50.024809 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-28908350-64c9-40f8-9fc7-8d905e63cf30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:50.090758 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1048/2099] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:06:50.102762 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d510116-99b2-4c83-83f7-25ea99348dca admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:50.104853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5d510116-99b2-4c83-83f7-25ea99348dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:50.164500 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d510116-99b2-4c83-83f7-25ea99348dca admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:50.168151 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ae7c8-18c6-4534-b767-e78031fb7a4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:50.168764 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ae7c8-18c6-4534-b767-e78031fb7a4b service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:50.169048 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ae7c8-18c6-4534-b767-e78031fb7a4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:50.169346 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ae7c8-18c6-4534-b767-e78031fb7a4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:50.179136 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ae7c8-18c6-4534-b767-e78031fb7a4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:50.180847 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ae7c8-18c6-4534-b767-e78031fb7a4b service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:50.194880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1052/2100] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:06:50.977902 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361f4dd-8744-43c1-af9d-8d98857c23fd None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:50.978429 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361f4dd-8744-43c1-af9d-8d98857c23fd None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:50.978429 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361f4dd-8744-43c1-af9d-8d98857c23fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:51.025470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-6361f4dd-8744-43c1-af9d-8d98857c23fd None None] MFA Rules not processed for user `375bce18b1f54a6b87978a2574f87b51`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:06:51.026841 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6361f4dd-8744-43c1-af9d-8d98857c23fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:51.066345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:06:51.134693 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1049/2101] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:06:50 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:06:51.139581 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ae7c8-18c6-4534-b767-e78031fb7a4b service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:51.145370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:06:51.155989 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ae7c8-18c6-4534-b767-e78031fb7a4b service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:51.162174 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e31c5-f6e0-41a5-9168-c5d7f5fc5e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:51.162843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e31c5-f6e0-41a5-9168-c5d7f5fc5e08 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:51.163363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e31c5-f6e0-41a5-9168-c5d7f5fc5e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:51.163741 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e31c5-f6e0-41a5-9168-c5d7f5fc5e08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:51.164547 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e31c5-f6e0-41a5-9168-c5d7f5fc5e08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:51.166631 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e31c5-f6e0-41a5-9168-c5d7f5fc5e08 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:51.202971 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1053/2102] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:06:51 2022] POST /identity/v3/projects => generated 403 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:06:51.206794 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361f4dd-8744-43c1-af9d-8d98857c23fd None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:51.218514 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361f4dd-8744-43c1-af9d-8d98857c23fd None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:51.296749 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08015ff8-a292-459a-92e8-1afd20f3b83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:51.297448 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08015ff8-a292-459a-92e8-1afd20f3b83e admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:51.297861 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08015ff8-a292-459a-92e8-1afd20f3b83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:51.297981 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08015ff8-a292-459a-92e8-1afd20f3b83e admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:51.299288 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08015ff8-a292-459a-92e8-1afd20f3b83e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:51.301164 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08015ff8-a292-459a-92e8-1afd20f3b83e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:51.325030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1050/2103] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:06:51 2022] POST /identity/v3/users => generated 308 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:06:51.326209 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e31c5-f6e0-41a5-9168-c5d7f5fc5e08 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:51.338633 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e31c5-f6e0-41a5-9168-c5d7f5fc5e08 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:51.343266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd8863-5647-413e-b1cd-5f8eb0645919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:51.343903 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd8863-5647-413e-b1cd-5f8eb0645919 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:51.344187 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd8863-5647-413e-b1cd-5f8eb0645919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:51.344463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd8863-5647-413e-b1cd-5f8eb0645919 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:51.345110 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd8863-5647-413e-b1cd-5f8eb0645919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:51.347234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd8863-5647-413e-b1cd-5f8eb0645919 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:51.357913 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1054/2104] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:06:51.362872 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08015ff8-a292-459a-92e8-1afd20f3b83e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:51.372919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08015ff8-a292-459a-92e8-1afd20f3b83e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:51.377622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745f018-1a46-48a6-bc44-a4774a1f2393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:51.378282 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745f018-1a46-48a6-bc44-a4774a1f2393 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:51.378569 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745f018-1a46-48a6-bc44-a4774a1f2393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:51.378853 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745f018-1a46-48a6-bc44-a4774a1f2393 admin admin] PATH_INFO: `/v3/projects/e5b1d3fefc34457da2dddccf502c3239/users/145da654040f4bb0ad43edebfdac7b10/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:51.394460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745f018-1a46-48a6-bc44-a4774a1f2393 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5b1d3fefc34457da2dddccf502c3239, user_id=145da654040f4bb0ad43edebfdac7b10, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:51.397177 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745f018-1a46-48a6-bc44-a4774a1f2393 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:51.411618 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1051/2105] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:06:51 2022] PUT /identity/v3/projects/e5b1d3fefc34457da2dddccf502c3239/users/145da654040f4bb0ad43edebfdac7b10/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:06:51.415645 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd8863-5647-413e-b1cd-5f8eb0645919 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:51.426512 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd8863-5647-413e-b1cd-5f8eb0645919 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:51.484736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef6a57-5c6c-4f29-9377-21ef8fddb148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:51.485371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef6a57-5c6c-4f29-9377-21ef8fddb148 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:51.485650 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef6a57-5c6c-4f29-9377-21ef8fddb148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:51.486007 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef6a57-5c6c-4f29-9377-21ef8fddb148 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:51.486855 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef6a57-5c6c-4f29-9377-21ef8fddb148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:51.488527 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef6a57-5c6c-4f29-9377-21ef8fddb148 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:51.499417 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1055/2106] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:06:51.503870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745f018-1a46-48a6-bc44-a4774a1f2393 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:51.514675 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745f018-1a46-48a6-bc44-a4774a1f2393 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:51.520223 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f085f-cbdf-463b-a2b4-603e508e2028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:51.521000 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f085f-cbdf-463b-a2b4-603e508e2028 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:51.521286 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f085f-cbdf-463b-a2b4-603e508e2028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:51.521567 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f085f-cbdf-463b-a2b4-603e508e2028 admin admin] PATH_INFO: `/v3/projects/e5b1d3fefc34457da2dddccf502c3239/users/145da654040f4bb0ad43edebfdac7b10/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:51.523533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6f085f-cbdf-463b-a2b4-603e508e2028 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5b1d3fefc34457da2dddccf502c3239, user_id=145da654040f4bb0ad43edebfdac7b10, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:51.526232 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6f085f-cbdf-463b-a2b4-603e508e2028 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:51.539621 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1052/2107] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:06:51 2022] PUT /identity/v3/projects/e5b1d3fefc34457da2dddccf502c3239/users/145da654040f4bb0ad43edebfdac7b10/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:06:52.363296 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585b7e1-4abb-4ffb-9688-a51b32cb8b2f None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:52.363895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585b7e1-4abb-4ffb-9688-a51b32cb8b2f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:52.364256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585b7e1-4abb-4ffb-9688-a51b32cb8b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:52.403844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-b585b7e1-4abb-4ffb-9688-a51b32cb8b2f None None] MFA Rules not processed for user `145da654040f4bb0ad43edebfdac7b10`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:06:52.405622 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b585b7e1-4abb-4ffb-9688-a51b32cb8b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:52.499062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1056/2108] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:06:52 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:06:53.210905 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50bfeb-67be-409c-b638-f05f6ae2ae99 None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:53.211454 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50bfeb-67be-409c-b638-f05f6ae2ae99 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:53.211917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50bfeb-67be-409c-b638-f05f6ae2ae99 None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:53.214396 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1053/2109] 10.222.0.42 () {54 vars in 921 bytes} [Wed Sep 14 01:06:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:06:53.223716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e1c90-82d3-4d21-a00d-2cc551fb46be None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:53.224048 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e1c90-82d3-4d21-a00d-2cc551fb46be None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:53.224434 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e1c90-82d3-4d21-a00d-2cc551fb46be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:53.265090 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-7c5e1c90-82d3-4d21-a00d-2cc551fb46be None None] MFA Rules not processed for user `18798838de284c84b5b838b3530a2c43`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:06:53.266363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7c5e1c90-82d3-4d21-a00d-2cc551fb46be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:53.349048 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:06:53.372984 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1057/2110] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Sep 14 01:06:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:06:53.384622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50bfeb-67be-409c-b638-f05f6ae2ae99 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:53.386813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fd50bfeb-67be-409c-b638-f05f6ae2ae99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:53.406020 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50bfeb-67be-409c-b638-f05f6ae2ae99 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:53.413323 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd3f8d-579d-42f6-80e4-ac148884e77a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18798838de284c84b5b838b3530a2c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:53.414009 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd3f8d-579d-42f6-80e4-ac148884e77a service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:53.414353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd3f8d-579d-42f6-80e4-ac148884e77a service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:53.414689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd3f8d-579d-42f6-80e4-ac148884e77a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:53.426267 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd3f8d-579d-42f6-80e4-ac148884e77a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:53.428258 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd3f8d-579d-42f6-80e4-ac148884e77a service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:53.441185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1054/2111] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:06:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:06:54.419393 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e1c90-82d3-4d21-a00d-2cc551fb46be None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:54.523734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e1c90-82d3-4d21-a00d-2cc551fb46be None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:54.529163 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af8bd9-e366-4edb-9d04-0f21bf5d6072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:54.529818 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af8bd9-e366-4edb-9d04-0f21bf5d6072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:54.530110 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af8bd9-e366-4edb-9d04-0f21bf5d6072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:54.530388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af8bd9-e366-4edb-9d04-0f21bf5d6072 admin admin] PATH_INFO: `/v3/users/375bce18b1f54a6b87978a2574f87b51` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:54.532097 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af8bd9-e366-4edb-9d04-0f21bf5d6072 admin admin] RBAC: Authorizing `identity:delete_user(user_id=375bce18b1f54a6b87978a2574f87b51)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:54.535200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af8bd9-e366-4edb-9d04-0f21bf5d6072 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:54.595627 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e9af8bd9-e366-4edb-9d04-0f21bf5d6072 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '375bce18b1f54a6b87978a2574f87b51'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:54.596535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e9af8bd9-e366-4edb-9d04-0f21bf5d6072 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '375bce18b1f54a6b87978a2574f87b51'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:54.605447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e9af8bd9-e366-4edb-9d04-0f21bf5d6072 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '375bce18b1f54a6b87978a2574f87b51'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:54.615478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e9af8bd9-e366-4edb-9d04-0f21bf5d6072 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '375bce18b1f54a6b87978a2574f87b51'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:54.616331 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e9af8bd9-e366-4edb-9d04-0f21bf5d6072 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '375bce18b1f54a6b87978a2574f87b51'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:54.638979 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1058/2112] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:06:54 2022] DELETE /identity/v3/users/375bce18b1f54a6b87978a2574f87b51 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:06:54.871137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd3f8d-579d-42f6-80e4-ac148884e77a service placement] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:54.892935 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd3f8d-579d-42f6-80e4-ac148884e77a service placement] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:55.002981 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55588592-7a5c-4e89-b0ab-95d739764ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:55.003668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55588592-7a5c-4e89-b0ab-95d739764ee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:55.004135 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55588592-7a5c-4e89-b0ab-95d739764ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:55.004629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55588592-7a5c-4e89-b0ab-95d739764ee0 admin admin] PATH_INFO: `/v3/projects/fb2f1550baa24e66a643c060fa34f8ed` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:55.006197 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55588592-7a5c-4e89-b0ab-95d739764ee0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb2f1550baa24e66a643c060fa34f8ed)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:55.008651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55588592-7a5c-4e89-b0ab-95d739764ee0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:55.011991 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:06:55.070893 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-55588592-7a5c-4e89-b0ab-95d739764ee0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb2f1550baa24e66a643c060fa34f8ed', 'request_id': 'req-55588592-7a5c-4e89-b0ab-95d739764ee0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:55.079514 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-55588592-7a5c-4e89-b0ab-95d739764ee0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb2f1550baa24e66a643c060fa34f8ed', 'request_id': 'req-55588592-7a5c-4e89-b0ab-95d739764ee0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:55.086743 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-55588592-7a5c-4e89-b0ab-95d739764ee0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb2f1550baa24e66a643c060fa34f8ed', 'request_id': 'req-55588592-7a5c-4e89-b0ab-95d739764ee0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:55.100651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-55588592-7a5c-4e89-b0ab-95d739764ee0 admin admin] The token cache is being invalidate because project fb2f1550baa24e66a643c060fa34f8ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:06:55.100950 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-55588592-7a5c-4e89-b0ab-95d739764ee0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:55.101615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-55588592-7a5c-4e89-b0ab-95d739764ee0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:55.103200 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1055/2113] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:06:54 2022] DELETE /identity/v3/projects/fb2f1550baa24e66a643c060fa34f8ed => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:06:55.107231 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af8bd9-e366-4edb-9d04-0f21bf5d6072 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:55.108768 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e9af8bd9-e366-4edb-9d04-0f21bf5d6072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:55.122601 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af8bd9-e366-4edb-9d04-0f21bf5d6072 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:55.204965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d607d4-4c90-455e-b751-6c4d484cf1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:55.205886 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d607d4-4c90-455e-b751-6c4d484cf1d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:55.206274 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d607d4-4c90-455e-b751-6c4d484cf1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:55.206881 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d607d4-4c90-455e-b751-6c4d484cf1d9 admin admin] PATH_INFO: `/v3/users/145da654040f4bb0ad43edebfdac7b10` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:55.208409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d607d4-4c90-455e-b751-6c4d484cf1d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=145da654040f4bb0ad43edebfdac7b10)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:55.210348 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d607d4-4c90-455e-b751-6c4d484cf1d9 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:55.220309 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:06:55.292308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a2d607d4-4c90-455e-b751-6c4d484cf1d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '145da654040f4bb0ad43edebfdac7b10'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:55.293081 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a2d607d4-4c90-455e-b751-6c4d484cf1d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '145da654040f4bb0ad43edebfdac7b10'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:55.300756 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a2d607d4-4c90-455e-b751-6c4d484cf1d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '145da654040f4bb0ad43edebfdac7b10'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:55.309360 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a2d607d4-4c90-455e-b751-6c4d484cf1d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '145da654040f4bb0ad43edebfdac7b10'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:55.310075 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a2d607d4-4c90-455e-b751-6c4d484cf1d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '145da654040f4bb0ad43edebfdac7b10'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:55.330520 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1059/2114] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:06:55 2022] DELETE /identity/v3/users/145da654040f4bb0ad43edebfdac7b10 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:06:55.516843 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55588592-7a5c-4e89-b0ab-95d739764ee0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:55.523634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-55588592-7a5c-4e89-b0ab-95d739764ee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:55.546463 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55588592-7a5c-4e89-b0ab-95d739764ee0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:55.612415 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712e9d3-b5c8-440b-8baf-4c725de715e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:55.613114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712e9d3-b5c8-440b-8baf-4c725de715e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:55.613393 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712e9d3-b5c8-440b-8baf-4c725de715e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:55.613662 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712e9d3-b5c8-440b-8baf-4c725de715e3 admin admin] PATH_INFO: `/v3/projects/e5b1d3fefc34457da2dddccf502c3239` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:55.614808 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0712e9d3-b5c8-440b-8baf-4c725de715e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5b1d3fefc34457da2dddccf502c3239)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:55.616848 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0712e9d3-b5c8-440b-8baf-4c725de715e3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:55.668736 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0712e9d3-b5c8-440b-8baf-4c725de715e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5b1d3fefc34457da2dddccf502c3239', 'request_id': 'req-0712e9d3-b5c8-440b-8baf-4c725de715e3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:55.676069 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0712e9d3-b5c8-440b-8baf-4c725de715e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5b1d3fefc34457da2dddccf502c3239', 'request_id': 'req-0712e9d3-b5c8-440b-8baf-4c725de715e3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:55.682003 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0712e9d3-b5c8-440b-8baf-4c725de715e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5b1d3fefc34457da2dddccf502c3239', 'request_id': 'req-0712e9d3-b5c8-440b-8baf-4c725de715e3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:55.695865 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0712e9d3-b5c8-440b-8baf-4c725de715e3 admin admin] The token cache is being invalidate because project e5b1d3fefc34457da2dddccf502c3239 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:06:55.696086 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0712e9d3-b5c8-440b-8baf-4c725de715e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:55.696774 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0712e9d3-b5c8-440b-8baf-4c725de715e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:06:55.698544 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1056/2115] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:06:55 2022] DELETE /identity/v3/projects/e5b1d3fefc34457da2dddccf502c3239 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:06:55.703847 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4ebf0-74cb-4f7f-985e-9754616be4bf None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:55.704361 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4ebf0-74cb-4f7f-985e-9754616be4bf None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:55.705060 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4ebf0-74cb-4f7f-985e-9754616be4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:55.726871 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-fcd4ebf0-74cb-4f7f-985e-9754616be4bf None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:06:55.728333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-fcd4ebf0-74cb-4f7f-985e-9754616be4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:55.792129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1060/2116] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:06:55.800227 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d40b1-1bb8-4cae-b1ca-419c6c34943e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:55.800570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d40b1-1bb8-4cae-b1ca-419c6c34943e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:55.800816 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d40b1-1bb8-4cae-b1ca-419c6c34943e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:55.821181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4a4d40b1-1bb8-4cae-b1ca-419c6c34943e None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:06:55.822373 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4a4d40b1-1bb8-4cae-b1ca-419c6c34943e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:55.830997 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1057/2117] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:06:55.835393 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4ebf0-74cb-4f7f-985e-9754616be4bf None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:55.845060 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4ebf0-74cb-4f7f-985e-9754616be4bf None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:55.849994 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face8af5-afec-4968-904b-e7f541e38d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:55.850598 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face8af5-afec-4968-904b-e7f541e38d36 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:55.850887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face8af5-afec-4968-904b-e7f541e38d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:55.851163 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face8af5-afec-4968-904b-e7f541e38d36 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:55.851853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face8af5-afec-4968-904b-e7f541e38d36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:06:55.852160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face8af5-afec-4968-904b-e7f541e38d36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:55.853764 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face8af5-afec-4968-904b-e7f541e38d36 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:55.865572 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1061/2118] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:06:55.870079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d40b1-1bb8-4cae-b1ca-419c6c34943e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:55.880597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d40b1-1bb8-4cae-b1ca-419c6c34943e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:55.885260 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22e6c6-6ffb-4bd9-84f1-4850ccad2967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:55.885859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22e6c6-6ffb-4bd9-84f1-4850ccad2967 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:55.886141 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22e6c6-6ffb-4bd9-84f1-4850ccad2967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:55.886415 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22e6c6-6ffb-4bd9-84f1-4850ccad2967 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:55.887048 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d22e6c6-6ffb-4bd9-84f1-4850ccad2967 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:55.889291 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d22e6c6-6ffb-4bd9-84f1-4850ccad2967 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:55.905276 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:06:55.922017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1058/2119] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:06:55 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:06:55.926182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face8af5-afec-4968-904b-e7f541e38d36 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:55.936211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face8af5-afec-4968-904b-e7f541e38d36 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:55.992781 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3f07d-43d0-4a7b-8302-d8160c816daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:55.993317 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3f07d-43d0-4a7b-8302-d8160c816daf admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:55.993603 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3f07d-43d0-4a7b-8302-d8160c816daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:55.993881 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3f07d-43d0-4a7b-8302-d8160c816daf admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:55.994519 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3f07d-43d0-4a7b-8302-d8160c816daf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:55.996522 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3f07d-43d0-4a7b-8302-d8160c816daf admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:56.019016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1062/2120] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:06:55 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:06:56.023253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22e6c6-6ffb-4bd9-84f1-4850ccad2967 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:56.034443 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22e6c6-6ffb-4bd9-84f1-4850ccad2967 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:56.039720 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42b94c-3e6f-49fc-9902-d9b4f8fb7e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:56.040152 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42b94c-3e6f-49fc-9902-d9b4f8fb7e08 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:56.040436 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42b94c-3e6f-49fc-9902-d9b4f8fb7e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:56.040733 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42b94c-3e6f-49fc-9902-d9b4f8fb7e08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:56.041361 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42b94c-3e6f-49fc-9902-d9b4f8fb7e08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:56.042968 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42b94c-3e6f-49fc-9902-d9b4f8fb7e08 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:56.053336 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1059/2121] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:06:56.057454 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3f07d-43d0-4a7b-8302-d8160c816daf admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:56.062634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:06:56.069406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3f07d-43d0-4a7b-8302-d8160c816daf admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:56.075018 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9dd9c-ba68-4c43-b0e7-feadafe298ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:56.075661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9dd9c-ba68-4c43-b0e7-feadafe298ac admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:56.075951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9dd9c-ba68-4c43-b0e7-feadafe298ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:56.076228 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9dd9c-ba68-4c43-b0e7-feadafe298ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:56.076872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9dd9c-ba68-4c43-b0e7-feadafe298ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:56.078467 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9dd9c-ba68-4c43-b0e7-feadafe298ac admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:56.089831 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1063/2122] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:06:56.094225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42b94c-3e6f-49fc-9902-d9b4f8fb7e08 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:56.104555 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42b94c-3e6f-49fc-9902-d9b4f8fb7e08 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:56.109471 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8327bc-4abc-4d7b-aa0c-7635be188812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:56.110117 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8327bc-4abc-4d7b-aa0c-7635be188812 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:56.110395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8327bc-4abc-4d7b-aa0c-7635be188812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:56.111173 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8327bc-4abc-4d7b-aa0c-7635be188812 admin admin] PATH_INFO: `/v3/projects/f1f386c2fe624e4b96ab4f4e95780db7/users/e884b139fe46471c985cf34e335d53ed/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:56.127200 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8327bc-4abc-4d7b-aa0c-7635be188812 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1f386c2fe624e4b96ab4f4e95780db7, user_id=e884b139fe46471c985cf34e335d53ed, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:56.129941 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8327bc-4abc-4d7b-aa0c-7635be188812 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:56.143832 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1060/2123] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:06:56 2022] PUT /identity/v3/projects/f1f386c2fe624e4b96ab4f4e95780db7/users/e884b139fe46471c985cf34e335d53ed/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:06:56.154388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8ec75-403f-41a1-bd1f-1a570314943a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:56.154677 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8ec75-403f-41a1-bd1f-1a570314943a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:56.154957 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8ec75-403f-41a1-bd1f-1a570314943a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:56.183499 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-c1b8ec75-403f-41a1-bd1f-1a570314943a None None] MFA Rules not processed for user `e884b139fe46471c985cf34e335d53ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:06:56.184741 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c1b8ec75-403f-41a1-bd1f-1a570314943a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:56.252099 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1064/2124] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:06:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:06:56.257058 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8327bc-4abc-4d7b-aa0c-7635be188812 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:56.322674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8327bc-4abc-4d7b-aa0c-7635be188812 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:56.327376 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab273ff-d074-440d-9103-2cbe2592a6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:56.328020 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab273ff-d074-440d-9103-2cbe2592a6b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:56.328305 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab273ff-d074-440d-9103-2cbe2592a6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:56.328584 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab273ff-d074-440d-9103-2cbe2592a6b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:56.329237 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab273ff-d074-440d-9103-2cbe2592a6b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:56.331231 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab273ff-d074-440d-9103-2cbe2592a6b9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:56.359342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1061/2125] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:06:56 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:06:56.364253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8ec75-403f-41a1-bd1f-1a570314943a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:56.375642 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8ec75-403f-41a1-bd1f-1a570314943a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:56.439715 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4d0a2-8962-4a28-a4e1-4bca131f27e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:56.440481 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4d0a2-8962-4a28-a4e1-4bca131f27e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:56.440779 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4d0a2-8962-4a28-a4e1-4bca131f27e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:56.441122 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4d0a2-8962-4a28-a4e1-4bca131f27e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:56.441949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4d0a2-8962-4a28-a4e1-4bca131f27e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:56.444188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4d0a2-8962-4a28-a4e1-4bca131f27e6 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:56.471910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1065/2126] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:06:56 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:06:56.473183 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab273ff-d074-440d-9103-2cbe2592a6b9 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:56.486799 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab273ff-d074-440d-9103-2cbe2592a6b9 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:56.492687 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb151886-bfba-41da-ab10-d65f76382a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:56.493355 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb151886-bfba-41da-ab10-d65f76382a24 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:56.493644 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb151886-bfba-41da-ab10-d65f76382a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:56.493922 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb151886-bfba-41da-ab10-d65f76382a24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:56.494576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb151886-bfba-41da-ab10-d65f76382a24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:56.496287 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb151886-bfba-41da-ab10-d65f76382a24 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:56.512099 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1062/2127] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:06:56.513292 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4d0a2-8962-4a28-a4e1-4bca131f27e6 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:56.524433 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4d0a2-8962-4a28-a4e1-4bca131f27e6 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:56.529493 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea89686-4111-4cab-bc34-b836fb719494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:56.530199 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea89686-4111-4cab-bc34-b836fb719494 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:56.530497 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea89686-4111-4cab-bc34-b836fb719494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:56.530778 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea89686-4111-4cab-bc34-b836fb719494 admin admin] PATH_INFO: `/v3/projects/c67abe6cab794ea68f412d13d2fb4f6e/users/a8bf176f382e4b928309949dfa009541/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:56.548796 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea89686-4111-4cab-bc34-b836fb719494 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c67abe6cab794ea68f412d13d2fb4f6e, user_id=a8bf176f382e4b928309949dfa009541, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:56.551518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea89686-4111-4cab-bc34-b836fb719494 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:56.565912 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1066/2128] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:06:56 2022] PUT /identity/v3/projects/c67abe6cab794ea68f412d13d2fb4f6e/users/a8bf176f382e4b928309949dfa009541/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:06:56.570152 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb151886-bfba-41da-ab10-d65f76382a24 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:56.582094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb151886-bfba-41da-ab10-d65f76382a24 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:56.641393 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b362e-ba84-4c32-b8a0-11a61ba67ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:56.642320 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b362e-ba84-4c32-b8a0-11a61ba67ecd admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:56.642596 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b362e-ba84-4c32-b8a0-11a61ba67ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:56.642867 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b362e-ba84-4c32-b8a0-11a61ba67ecd admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:56.643505 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b362e-ba84-4c32-b8a0-11a61ba67ecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:56.645129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b362e-ba84-4c32-b8a0-11a61ba67ecd admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:56.655076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1063/2129] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:06:56.660388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea89686-4111-4cab-bc34-b836fb719494 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:56.671988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea89686-4111-4cab-bc34-b836fb719494 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:56.677958 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f66963-3679-4b38-a19c-4c50998d19cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:56.678795 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f66963-3679-4b38-a19c-4c50998d19cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:56.679180 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f66963-3679-4b38-a19c-4c50998d19cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:56.679536 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f66963-3679-4b38-a19c-4c50998d19cc admin admin] PATH_INFO: `/v3/projects/c67abe6cab794ea68f412d13d2fb4f6e/users/a8bf176f382e4b928309949dfa009541/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:56.681698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f66963-3679-4b38-a19c-4c50998d19cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c67abe6cab794ea68f412d13d2fb4f6e, user_id=a8bf176f382e4b928309949dfa009541, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:56.684932 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f66963-3679-4b38-a19c-4c50998d19cc admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:56.696946 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1067/2130] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:06:56 2022] PUT /identity/v3/projects/c67abe6cab794ea68f412d13d2fb4f6e/users/a8bf176f382e4b928309949dfa009541/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:06:56.709725 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13b5be-b401-40cf-b735-abd640405c8b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:56.710043 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13b5be-b401-40cf-b735-abd640405c8b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:56.710322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13b5be-b401-40cf-b735-abd640405c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:56.741545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ae13b5be-b401-40cf-b735-abd640405c8b None None] MFA Rules not processed for user `a8bf176f382e4b928309949dfa009541`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:06:56.742763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ae13b5be-b401-40cf-b735-abd640405c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:56.762247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:06:56.820193 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1064/2131] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:06:56 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:06:56.831867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f66963-3679-4b38-a19c-4c50998d19cc admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:06:56.833684 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-70f66963-3679-4b38-a19c-4c50998d19cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:06:56.870652 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:06:56.896306 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f66963-3679-4b38-a19c-4c50998d19cc admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:06:56.901098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebf358-e76f-45f0-adc0-54216ff0c76b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:06:56.901740 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebf358-e76f-45f0-adc0-54216ff0c76b service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:06:56.902111 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebf358-e76f-45f0-adc0-54216ff0c76b service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:06:56.902387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebf358-e76f-45f0-adc0-54216ff0c76b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:06:56.963727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebf358-e76f-45f0-adc0-54216ff0c76b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:06:56.965191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebf358-e76f-45f0-adc0-54216ff0c76b service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:06:56.974828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1068/2132] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:06:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:07:00.857509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb7ecc-c957-4c8e-beba-36af56a37b67 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:00.857978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb7ecc-c957-4c8e-beba-36af56a37b67 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:00.858128 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb7ecc-c957-4c8e-beba-36af56a37b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:00.878365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-dceb7ecc-c957-4c8e-beba-36af56a37b67 None None] MFA Rules not processed for user `ceccc909529042539dd2d5015058d8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:07:00.879609 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dceb7ecc-c957-4c8e-beba-36af56a37b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:07:00.941719 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1065/2133] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 01:07:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:07:00.986569 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebf358-e76f-45f0-adc0-54216ff0c76b service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:00.988169 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0eebf358-e76f-45f0-adc0-54216ff0c76b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:07:01.066078 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebf358-e76f-45f0-adc0-54216ff0c76b service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:01.072554 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd36e9e-1e7d-4c2e-98a5-4e5c52737b3b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e52776d9b4894af9b85a24136a8255e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:01.073297 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd36e9e-1e7d-4c2e-98a5-4e5c52737b3b service swift] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:01.073672 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd36e9e-1e7d-4c2e-98a5-4e5c52737b3b service swift] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:01.073929 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd36e9e-1e7d-4c2e-98a5-4e5c52737b3b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:01.083257 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd36e9e-1e7d-4c2e-98a5-4e5c52737b3b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:01.084894 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd36e9e-1e7d-4c2e-98a5-4e5c52737b3b service swift] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:01.094645 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1069/2134] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 01:07:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 14 01:07:08.112262 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb7ecc-c957-4c8e-beba-36af56a37b67 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:08.123592 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb7ecc-c957-4c8e-beba-36af56a37b67 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:08.125701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfeb33a-5b27-4fb2-9464-de41d199cdbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:08.126561 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfeb33a-5b27-4fb2-9464-de41d199cdbb service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:08.126734 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfeb33a-5b27-4fb2-9464-de41d199cdbb service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:08.127015 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfeb33a-5b27-4fb2-9464-de41d199cdbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:08.137831 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfeb33a-5b27-4fb2-9464-de41d199cdbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:08.139850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfeb33a-5b27-4fb2-9464-de41d199cdbb service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:08.152515 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1066/2135] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:07:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:07:10.096590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1780b-bf17-459f-b6cc-8fa74bcbb0b8 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:10.097268 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1780b-bf17-459f-b6cc-8fa74bcbb0b8 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:10.097368 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1780b-bf17-459f-b6cc-8fa74bcbb0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:10.156045 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-fce1780b-bf17-459f-b6cc-8fa74bcbb0b8 None None] MFA Rules not processed for user `bf73eca572454b88843785d791f0b471`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:07:10.158589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-fce1780b-bf17-459f-b6cc-8fa74bcbb0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:07:10.255623 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1070/2136] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:07:10 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:07:18.147617 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=52 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:07:20.246929 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=90 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:07:23.131857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255db615-1453-406b-b5e9-449423b631e1 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:23.132482 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255db615-1453-406b-b5e9-449423b631e1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:23.132482 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255db615-1453-406b-b5e9-449423b631e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:23.152668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-255db615-1453-406b-b5e9-449423b631e1 None None] MFA Rules not processed for user `ceccc909529042539dd2d5015058d8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:07:23.154015 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-255db615-1453-406b-b5e9-449423b631e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:07:23.162283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1067/2137] 10.222.0.42 () {58 vars in 967 bytes} [Wed Sep 14 01:07:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:07:23.175632 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1780b-bf17-459f-b6cc-8fa74bcbb0b8 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:23.188526 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1780b-bf17-459f-b6cc-8fa74bcbb0b8 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:23.199282 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a061f-8039-42e0-aebf-607e642469e8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e52776d9b4894af9b85a24136a8255e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:23.200091 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a061f-8039-42e0-aebf-607e642469e8 service swift] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:23.200373 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a061f-8039-42e0-aebf-607e642469e8 service swift] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:23.200721 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a061f-8039-42e0-aebf-607e642469e8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:23.210788 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a061f-8039-42e0-aebf-607e642469e8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:23.212439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a061f-8039-42e0-aebf-607e642469e8 service swift] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:23.221133 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1071/2138] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 01:07:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 01:07:25.244512 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255db615-1453-406b-b5e9-449423b631e1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:25.246055 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-255db615-1453-406b-b5e9-449423b631e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:07:25.321369 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255db615-1453-406b-b5e9-449423b631e1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:25.325942 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc17f3c-1d75-43e6-a74f-26713fc3cc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:25.326576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc17f3c-1d75-43e6-a74f-26713fc3cc9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:25.326854 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc17f3c-1d75-43e6-a74f-26713fc3cc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:25.327233 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc17f3c-1d75-43e6-a74f-26713fc3cc9d admin admin] PATH_INFO: `/v3/users/9bca70e61b80499eb07a668e11534257` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:25.328866 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc17f3c-1d75-43e6-a74f-26713fc3cc9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bca70e61b80499eb07a668e11534257)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:25.331081 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc17f3c-1d75-43e6-a74f-26713fc3cc9d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:25.378461 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5cc17f3c-1d75-43e6-a74f-26713fc3cc9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bca70e61b80499eb07a668e11534257'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:25.379126 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5cc17f3c-1d75-43e6-a74f-26713fc3cc9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bca70e61b80499eb07a668e11534257'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:25.386996 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5cc17f3c-1d75-43e6-a74f-26713fc3cc9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bca70e61b80499eb07a668e11534257'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:25.387610 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5cc17f3c-1d75-43e6-a74f-26713fc3cc9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bca70e61b80499eb07a668e11534257'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:25.396493 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5cc17f3c-1d75-43e6-a74f-26713fc3cc9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bca70e61b80499eb07a668e11534257'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:25.413975 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1068/2139] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:07:25 2022] DELETE /identity/v3/users/9bca70e61b80499eb07a668e11534257 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:07:25.587723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a061f-8039-42e0-aebf-607e642469e8 service swift] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:25.606767 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a061f-8039-42e0-aebf-607e642469e8 service swift] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:25.665684 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b988d3e4-c1f5-492c-9cf4-cceb67f76a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:25.666317 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988d3e4-c1f5-492c-9cf4-cceb67f76a06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:25.666603 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988d3e4-c1f5-492c-9cf4-cceb67f76a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:25.666883 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988d3e4-c1f5-492c-9cf4-cceb67f76a06 admin admin] PATH_INFO: `/v3/projects/6df5fe3a39fd4f9fb1ae104e43ec5529` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:25.668047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b988d3e4-c1f5-492c-9cf4-cceb67f76a06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6df5fe3a39fd4f9fb1ae104e43ec5529)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:25.670019 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b988d3e4-c1f5-492c-9cf4-cceb67f76a06 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:25.715993 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b988d3e4-c1f5-492c-9cf4-cceb67f76a06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6df5fe3a39fd4f9fb1ae104e43ec5529', 'request_id': 'req-b988d3e4-c1f5-492c-9cf4-cceb67f76a06'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:25.723310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b988d3e4-c1f5-492c-9cf4-cceb67f76a06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6df5fe3a39fd4f9fb1ae104e43ec5529', 'request_id': 'req-b988d3e4-c1f5-492c-9cf4-cceb67f76a06'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:25.729161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b988d3e4-c1f5-492c-9cf4-cceb67f76a06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6df5fe3a39fd4f9fb1ae104e43ec5529', 'request_id': 'req-b988d3e4-c1f5-492c-9cf4-cceb67f76a06'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:25.740437 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b988d3e4-c1f5-492c-9cf4-cceb67f76a06 admin admin] The token cache is being invalidate because project 6df5fe3a39fd4f9fb1ae104e43ec5529 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:07:25.740727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b988d3e4-c1f5-492c-9cf4-cceb67f76a06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:25.741374 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b988d3e4-c1f5-492c-9cf4-cceb67f76a06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:25.742858 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1072/2140] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:07:25 2022] DELETE /identity/v3/projects/6df5fe3a39fd4f9fb1ae104e43ec5529 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:07:25.747695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a2873-d093-4715-bc5c-074d901196f9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:25.748060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a2873-d093-4715-bc5c-074d901196f9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:25.748341 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a2873-d093-4715-bc5c-074d901196f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:25.769231 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0e8a2873-d093-4715-bc5c-074d901196f9 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:07:25.770463 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0e8a2873-d093-4715-bc5c-074d901196f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:07:25.829338 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1069/2141] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:07:25.836858 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489a402-e015-4748-a972-fbdc8f359257 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:25.837092 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489a402-e015-4748-a972-fbdc8f359257 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:25.837373 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489a402-e015-4748-a972-fbdc8f359257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:25.857108 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-2489a402-e015-4748-a972-fbdc8f359257 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:07:25.858317 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2489a402-e015-4748-a972-fbdc8f359257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:07:25.866277 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1073/2142] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:07:25.870421 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a2873-d093-4715-bc5c-074d901196f9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:25.880799 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a2873-d093-4715-bc5c-074d901196f9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:25.885373 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50956ea6-8edc-46da-9f15-ef619dfcd4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:25.885966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50956ea6-8edc-46da-9f15-ef619dfcd4f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:25.886241 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50956ea6-8edc-46da-9f15-ef619dfcd4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:25.886509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50956ea6-8edc-46da-9f15-ef619dfcd4f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:25.887180 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50956ea6-8edc-46da-9f15-ef619dfcd4f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:07:25.887480 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50956ea6-8edc-46da-9f15-ef619dfcd4f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:25.889349 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50956ea6-8edc-46da-9f15-ef619dfcd4f9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:25.898600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:07:25.902092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1070/2143] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:07:25.906793 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489a402-e015-4748-a972-fbdc8f359257 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:25.915919 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489a402-e015-4748-a972-fbdc8f359257 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:25.920489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29fae3e-4072-4512-a39f-33e50561e28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:25.921080 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fae3e-4072-4512-a39f-33e50561e28d admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:25.921362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fae3e-4072-4512-a39f-33e50561e28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:25.921663 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fae3e-4072-4512-a39f-33e50561e28d admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:25.922661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29fae3e-4072-4512-a39f-33e50561e28d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:25.924747 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29fae3e-4072-4512-a39f-33e50561e28d admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:25.951657 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1074/2144] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:07:25 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:07:25.956783 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50956ea6-8edc-46da-9f15-ef619dfcd4f9 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:25.966760 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50956ea6-8edc-46da-9f15-ef619dfcd4f9 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:26.022114 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce428f-5e1f-4073-9555-48a80e074bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:26.022715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce428f-5e1f-4073-9555-48a80e074bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:26.022999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce428f-5e1f-4073-9555-48a80e074bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:26.023274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce428f-5e1f-4073-9555-48a80e074bd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:26.023953 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce428f-5e1f-4073-9555-48a80e074bd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:26.025889 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce428f-5e1f-4073-9555-48a80e074bd0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:26.048105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1071/2145] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:07:25 2022] POST /identity/v3/users => generated 285 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:07:26.050920 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29fae3e-4072-4512-a39f-33e50561e28d admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:26.061307 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29fae3e-4072-4512-a39f-33e50561e28d admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:26.066813 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80189a-630b-4dcc-a032-63f1e2a6ba77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:26.067411 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80189a-630b-4dcc-a032-63f1e2a6ba77 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:26.067736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80189a-630b-4dcc-a032-63f1e2a6ba77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:26.068016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80189a-630b-4dcc-a032-63f1e2a6ba77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:26.068662 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80189a-630b-4dcc-a032-63f1e2a6ba77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:26.070248 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80189a-630b-4dcc-a032-63f1e2a6ba77 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:26.080282 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1075/2146] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:07:26.084183 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce428f-5e1f-4073-9555-48a80e074bd0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:26.094194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce428f-5e1f-4073-9555-48a80e074bd0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:26.098785 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac1433-30b7-4008-8a56-64966a2c1c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:26.099384 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac1433-30b7-4008-8a56-64966a2c1c7d admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:26.099710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac1433-30b7-4008-8a56-64966a2c1c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:26.099988 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac1433-30b7-4008-8a56-64966a2c1c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:26.100632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ac1433-30b7-4008-8a56-64966a2c1c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:26.102221 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ac1433-30b7-4008-8a56-64966a2c1c7d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:26.115418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1072/2147] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:07:26.118085 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80189a-630b-4dcc-a032-63f1e2a6ba77 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:26.128681 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80189a-630b-4dcc-a032-63f1e2a6ba77 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:26.134329 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9e90c-577a-4e57-adfb-264e82036ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:26.134986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9e90c-577a-4e57-adfb-264e82036ec6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:26.135269 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9e90c-577a-4e57-adfb-264e82036ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:26.135546 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9e90c-577a-4e57-adfb-264e82036ec6 admin admin] PATH_INFO: `/v3/projects/0b5606e8f3884720be78850662d9647f/users/35611aeb415a416799fbb2cd4a080afe/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:26.157346 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f9e90c-577a-4e57-adfb-264e82036ec6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b5606e8f3884720be78850662d9647f, user_id=35611aeb415a416799fbb2cd4a080afe, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:26.160306 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f9e90c-577a-4e57-adfb-264e82036ec6 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:26.174560 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1076/2148] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:07:26 2022] PUT /identity/v3/projects/0b5606e8f3884720be78850662d9647f/users/35611aeb415a416799fbb2cd4a080afe/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:07:26.185459 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3d3cb-501a-42a1-830e-bde4e8e32b09 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:26.185748 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3d3cb-501a-42a1-830e-bde4e8e32b09 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:26.186022 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3d3cb-501a-42a1-830e-bde4e8e32b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:26.218890 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e5c3d3cb-501a-42a1-830e-bde4e8e32b09 None None] MFA Rules not processed for user `35611aeb415a416799fbb2cd4a080afe`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:07:26.220135 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e5c3d3cb-501a-42a1-830e-bde4e8e32b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:07:26.283993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1073/2149] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:07:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:07:26.295578 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9e90c-577a-4e57-adfb-264e82036ec6 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:26.297756 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d3f9e90c-577a-4e57-adfb-264e82036ec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:07:26.313408 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:07:26.358653 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9e90c-577a-4e57-adfb-264e82036ec6 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:26.363147 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317341b2-7d60-4a15-9e79-b500990ba37f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89ff357209194276bc079d0c05898747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:26.363781 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317341b2-7d60-4a15-9e79-b500990ba37f service glance] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:26.364064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317341b2-7d60-4a15-9e79-b500990ba37f service glance] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:26.364345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317341b2-7d60-4a15-9e79-b500990ba37f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:26.372754 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317341b2-7d60-4a15-9e79-b500990ba37f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:26.374185 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317341b2-7d60-4a15-9e79-b500990ba37f service glance] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:26.382966 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1077/2150] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:07:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:07:36.281483 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:07:36.379020 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=20 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:07:51.525158 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3d3cb-501a-42a1-830e-bde4e8e32b09 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:51.526551 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e5c3d3cb-501a-42a1-830e-bde4e8e32b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:07:51.606127 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3d3cb-501a-42a1-830e-bde4e8e32b09 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:51.611762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6f514-c4a9-4031-b113-4f8937f8c4a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18798838de284c84b5b838b3530a2c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:51.612395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6f514-c4a9-4031-b113-4f8937f8c4a9 service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:51.612675 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6f514-c4a9-4031-b113-4f8937f8c4a9 service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:51.612948 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6f514-c4a9-4031-b113-4f8937f8c4a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:51.614718 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-04d6f514-c4a9-4031-b113-4f8937f8c4a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:07:51.628174 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d6f514-c4a9-4031-b113-4f8937f8c4a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:51.629792 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d6f514-c4a9-4031-b113-4f8937f8c4a9 service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:51.641092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1074/2151] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:07:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:07:55.275994 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317341b2-7d60-4a15-9e79-b500990ba37f service glance] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:55.277400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-317341b2-7d60-4a15-9e79-b500990ba37f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:07:55.356594 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317341b2-7d60-4a15-9e79-b500990ba37f service glance] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:55.361236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c555a-0c8b-4110-9e61-17985d63a8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:55.361864 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c555a-0c8b-4110-9e61-17985d63a8bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:55.362142 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c555a-0c8b-4110-9e61-17985d63a8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:55.362416 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c555a-0c8b-4110-9e61-17985d63a8bb admin admin] PATH_INFO: `/v3/users/e884b139fe46471c985cf34e335d53ed` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:55.364514 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c555a-0c8b-4110-9e61-17985d63a8bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e884b139fe46471c985cf34e335d53ed)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:55.366482 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c555a-0c8b-4110-9e61-17985d63a8bb admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:55.415665 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6d9c555a-0c8b-4110-9e61-17985d63a8bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e884b139fe46471c985cf34e335d53ed'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:55.416411 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6d9c555a-0c8b-4110-9e61-17985d63a8bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e884b139fe46471c985cf34e335d53ed'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:55.423972 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6d9c555a-0c8b-4110-9e61-17985d63a8bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e884b139fe46471c985cf34e335d53ed'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:55.432180 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6d9c555a-0c8b-4110-9e61-17985d63a8bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e884b139fe46471c985cf34e335d53ed'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:55.433156 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6d9c555a-0c8b-4110-9e61-17985d63a8bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e884b139fe46471c985cf34e335d53ed'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:55.455497 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1078/2152] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:07:55 2022] DELETE /identity/v3/users/e884b139fe46471c985cf34e335d53ed => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:07:55.460606 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fadd95-78ee-4261-9423-ee206d64a5a2 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:55.461129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fadd95-78ee-4261-9423-ee206d64a5a2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:55.461129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fadd95-78ee-4261-9423-ee206d64a5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:55.482655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-74fadd95-78ee-4261-9423-ee206d64a5a2 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:07:55.483888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-74fadd95-78ee-4261-9423-ee206d64a5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:07:55.546823 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1075/2153] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:07:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:07:55.557067 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c555a-0c8b-4110-9e61-17985d63a8bb admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:55.558867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6d9c555a-0c8b-4110-9e61-17985d63a8bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:07:55.618763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c555a-0c8b-4110-9e61-17985d63a8bb admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:55.622493 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0d398-657d-45e2-9053-16240092ba19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:55.623107 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0d398-657d-45e2-9053-16240092ba19 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:55.623392 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0d398-657d-45e2-9053-16240092ba19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:55.623702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0d398-657d-45e2-9053-16240092ba19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:55.633582 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed0d398-657d-45e2-9053-16240092ba19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:55.635029 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed0d398-657d-45e2-9053-16240092ba19 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:55.645037 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1079/2154] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:07:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:07:55.938817 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fadd95-78ee-4261-9423-ee206d64a5a2 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:55.940328 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-74fadd95-78ee-4261-9423-ee206d64a5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:07:55.955848 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fadd95-78ee-4261-9423-ee206d64a5a2 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:55.961676 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d1d1b-8cb4-4a6e-b871-5d701c01e8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:55.963032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d1d1b-8cb4-4a6e-b871-5d701c01e8b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:55.963547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d1d1b-8cb4-4a6e-b871-5d701c01e8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:55.963921 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d1d1b-8cb4-4a6e-b871-5d701c01e8b5 admin admin] PATH_INFO: `/v3/projects/f1f386c2fe624e4b96ab4f4e95780db7` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:55.965195 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d1d1b-8cb4-4a6e-b871-5d701c01e8b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1f386c2fe624e4b96ab4f4e95780db7)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:55.967573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d1d1b-8cb4-4a6e-b871-5d701c01e8b5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:56.021254 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1c8d1d1b-8cb4-4a6e-b871-5d701c01e8b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1f386c2fe624e4b96ab4f4e95780db7', 'request_id': 'req-1c8d1d1b-8cb4-4a6e-b871-5d701c01e8b5'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:56.031824 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1c8d1d1b-8cb4-4a6e-b871-5d701c01e8b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1f386c2fe624e4b96ab4f4e95780db7', 'request_id': 'req-1c8d1d1b-8cb4-4a6e-b871-5d701c01e8b5'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:56.038841 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1c8d1d1b-8cb4-4a6e-b871-5d701c01e8b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1f386c2fe624e4b96ab4f4e95780db7', 'request_id': 'req-1c8d1d1b-8cb4-4a6e-b871-5d701c01e8b5'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:56.051874 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1c8d1d1b-8cb4-4a6e-b871-5d701c01e8b5 admin admin] The token cache is being invalidate because project f1f386c2fe624e4b96ab4f4e95780db7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:07:56.052151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1c8d1d1b-8cb4-4a6e-b871-5d701c01e8b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:56.053034 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1c8d1d1b-8cb4-4a6e-b871-5d701c01e8b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:56.054548 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1076/2155] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:07:55 2022] DELETE /identity/v3/projects/f1f386c2fe624e4b96ab4f4e95780db7 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:07:56.058737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0d398-657d-45e2-9053-16240092ba19 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:56.069455 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0d398-657d-45e2-9053-16240092ba19 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:56.088909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:07:56.132369 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d4a66-7298-414f-8179-b67062127396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:56.133020 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d4a66-7298-414f-8179-b67062127396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:56.133308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d4a66-7298-414f-8179-b67062127396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:56.133585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d4a66-7298-414f-8179-b67062127396 admin admin] PATH_INFO: `/v3/users/a8bf176f382e4b928309949dfa009541` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:56.134741 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d4a66-7298-414f-8179-b67062127396 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8bf176f382e4b928309949dfa009541)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:56.136761 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d4a66-7298-414f-8179-b67062127396 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:56.184536 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d79d4a66-7298-414f-8179-b67062127396 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8bf176f382e4b928309949dfa009541'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:56.185189 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d79d4a66-7298-414f-8179-b67062127396 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8bf176f382e4b928309949dfa009541'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:56.194329 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d79d4a66-7298-414f-8179-b67062127396 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8bf176f382e4b928309949dfa009541'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:56.201926 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d79d4a66-7298-414f-8179-b67062127396 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8bf176f382e4b928309949dfa009541'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:56.202606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d79d4a66-7298-414f-8179-b67062127396 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8bf176f382e4b928309949dfa009541'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:56.220987 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1080/2156] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:07:56 2022] DELETE /identity/v3/users/a8bf176f382e4b928309949dfa009541 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:07:56.513980 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d1d1b-8cb4-4a6e-b871-5d701c01e8b5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:56.515427 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1c8d1d1b-8cb4-4a6e-b871-5d701c01e8b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:07:56.540138 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d1d1b-8cb4-4a6e-b871-5d701c01e8b5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:56.553093 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:07:56.615514 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b7af4a-1ff9-4140-a1e2-7d70a2f14c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:56.616444 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7af4a-1ff9-4140-a1e2-7d70a2f14c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:56.616724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7af4a-1ff9-4140-a1e2-7d70a2f14c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:56.616989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7af4a-1ff9-4140-a1e2-7d70a2f14c96 admin admin] PATH_INFO: `/v3/projects/c67abe6cab794ea68f412d13d2fb4f6e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:56.618466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b7af4a-1ff9-4140-a1e2-7d70a2f14c96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c67abe6cab794ea68f412d13d2fb4f6e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:56.620423 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b7af4a-1ff9-4140-a1e2-7d70a2f14c96 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:56.674542 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-04b7af4a-1ff9-4140-a1e2-7d70a2f14c96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c67abe6cab794ea68f412d13d2fb4f6e', 'request_id': 'req-04b7af4a-1ff9-4140-a1e2-7d70a2f14c96'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:56.683381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-04b7af4a-1ff9-4140-a1e2-7d70a2f14c96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c67abe6cab794ea68f412d13d2fb4f6e', 'request_id': 'req-04b7af4a-1ff9-4140-a1e2-7d70a2f14c96'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:56.690219 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-04b7af4a-1ff9-4140-a1e2-7d70a2f14c96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c67abe6cab794ea68f412d13d2fb4f6e', 'request_id': 'req-04b7af4a-1ff9-4140-a1e2-7d70a2f14c96'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:56.703252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-04b7af4a-1ff9-4140-a1e2-7d70a2f14c96 admin admin] The token cache is being invalidate because project c67abe6cab794ea68f412d13d2fb4f6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:07:56.703718 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-04b7af4a-1ff9-4140-a1e2-7d70a2f14c96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:56.704443 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-04b7af4a-1ff9-4140-a1e2-7d70a2f14c96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:07:56.707603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1077/2157] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:07:56 2022] DELETE /identity/v3/projects/c67abe6cab794ea68f412d13d2fb4f6e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:07:56.711350 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546be2f1-0fd8-408c-a495-6957656a60d7 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:56.711864 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546be2f1-0fd8-408c-a495-6957656a60d7 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:56.711971 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546be2f1-0fd8-408c-a495-6957656a60d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:56.733960 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-546be2f1-0fd8-408c-a495-6957656a60d7 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:07:56.734391 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-546be2f1-0fd8-408c-a495-6957656a60d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:07:56.825225 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1081/2158] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:07:56.834593 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab6915-588e-4165-bd78-a243f25dc072 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:56.834976 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab6915-588e-4165-bd78-a243f25dc072 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:56.835167 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab6915-588e-4165-bd78-a243f25dc072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:56.856474 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-90ab6915-588e-4165-bd78-a243f25dc072 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:07:56.857794 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-90ab6915-588e-4165-bd78-a243f25dc072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:07:56.868754 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1078/2159] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:07:56.873126 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546be2f1-0fd8-408c-a495-6957656a60d7 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:56.886344 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546be2f1-0fd8-408c-a495-6957656a60d7 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:56.892669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df47539-ce7d-421b-83ec-aafa2973d614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:56.893478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df47539-ce7d-421b-83ec-aafa2973d614 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:56.894048 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df47539-ce7d-421b-83ec-aafa2973d614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:56.894510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df47539-ce7d-421b-83ec-aafa2973d614 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:56.895419 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df47539-ce7d-421b-83ec-aafa2973d614 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:07:56.896281 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df47539-ce7d-421b-83ec-aafa2973d614 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:56.898083 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df47539-ce7d-421b-83ec-aafa2973d614 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:56.916309 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1082/2160] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:07:56.918694 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab6915-588e-4165-bd78-a243f25dc072 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:56.929212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab6915-588e-4165-bd78-a243f25dc072 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:56.934298 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9404fa-bb80-42ef-9202-f1893ad12a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:56.934908 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9404fa-bb80-42ef-9202-f1893ad12a5c admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:56.935188 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9404fa-bb80-42ef-9202-f1893ad12a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:56.935460 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9404fa-bb80-42ef-9202-f1893ad12a5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:56.936131 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9404fa-bb80-42ef-9202-f1893ad12a5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:56.938090 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9404fa-bb80-42ef-9202-f1893ad12a5c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:56.969860 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1079/2161] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:07:56 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:07:56.975209 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df47539-ce7d-421b-83ec-aafa2973d614 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:56.986099 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df47539-ce7d-421b-83ec-aafa2973d614 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:57.017470 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:07:57.064436 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258bfa2-ddd2-465e-888d-e4ab06488d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:57.065494 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258bfa2-ddd2-465e-888d-e4ab06488d06 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:57.065815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258bfa2-ddd2-465e-888d-e4ab06488d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:57.066097 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258bfa2-ddd2-465e-888d-e4ab06488d06 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:57.066764 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258bfa2-ddd2-465e-888d-e4ab06488d06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:57.069853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258bfa2-ddd2-465e-888d-e4ab06488d06 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:57.094097 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1083/2162] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:07:56 2022] POST /identity/v3/users => generated 285 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:07:57.098691 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9404fa-bb80-42ef-9202-f1893ad12a5c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:57.109211 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9404fa-bb80-42ef-9202-f1893ad12a5c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:57.114326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb90bc3-3edf-4384-9057-e3d262d8c51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:57.114930 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb90bc3-3edf-4384-9057-e3d262d8c51a admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:57.115233 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb90bc3-3edf-4384-9057-e3d262d8c51a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:57.115535 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb90bc3-3edf-4384-9057-e3d262d8c51a admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:57.116202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb90bc3-3edf-4384-9057-e3d262d8c51a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:57.117812 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb90bc3-3edf-4384-9057-e3d262d8c51a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:57.128787 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1080/2163] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:07:57.132753 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258bfa2-ddd2-465e-888d-e4ab06488d06 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:57.144203 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258bfa2-ddd2-465e-888d-e4ab06488d06 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:57.149918 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c7041-884a-487a-a741-ea2c79a6fcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:57.150561 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c7041-884a-487a-a741-ea2c79a6fcca admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:57.150846 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c7041-884a-487a-a741-ea2c79a6fcca admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:57.151124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c7041-884a-487a-a741-ea2c79a6fcca admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:57.151895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80c7041-884a-487a-a741-ea2c79a6fcca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:57.153514 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80c7041-884a-487a-a741-ea2c79a6fcca admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:57.165542 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1084/2164] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:07:57.169857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb90bc3-3edf-4384-9057-e3d262d8c51a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:57.181145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb90bc3-3edf-4384-9057-e3d262d8c51a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:57.185752 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444eb5f-d488-4192-8dec-913a527c50c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:57.186426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444eb5f-d488-4192-8dec-913a527c50c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:57.186707 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444eb5f-d488-4192-8dec-913a527c50c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:57.186984 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444eb5f-d488-4192-8dec-913a527c50c3 admin admin] PATH_INFO: `/v3/projects/1ca30fa38e20438a9315f2d63634cf5a/users/697ec9fdb3ef40238bb285a931bc0bce/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:57.203007 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444eb5f-d488-4192-8dec-913a527c50c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ca30fa38e20438a9315f2d63634cf5a, user_id=697ec9fdb3ef40238bb285a931bc0bce, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:57.205731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444eb5f-d488-4192-8dec-913a527c50c3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:57.219462 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1081/2165] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:07:57 2022] PUT /identity/v3/projects/1ca30fa38e20438a9315f2d63634cf5a/users/697ec9fdb3ef40238bb285a931bc0bce/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:07:57.231583 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d83308-fdc2-4c32-9778-3d2e016e84eb None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:57.231919 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d83308-fdc2-4c32-9778-3d2e016e84eb None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:57.232198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d83308-fdc2-4c32-9778-3d2e016e84eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:57.277819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-02d83308-fdc2-4c32-9778-3d2e016e84eb None None] MFA Rules not processed for user `697ec9fdb3ef40238bb285a931bc0bce`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:07:57.279041 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-02d83308-fdc2-4c32-9778-3d2e016e84eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:07:57.352938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1085/2166] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:07:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:07:57.357541 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444eb5f-d488-4192-8dec-913a527c50c3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:57.402213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:07:57.424674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444eb5f-d488-4192-8dec-913a527c50c3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:57.431033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d40f3-2774-4854-96b1-0a364bde127a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:57.431033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d40f3-2774-4854-96b1-0a364bde127a admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:57.431033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d40f3-2774-4854-96b1-0a364bde127a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:57.431033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d40f3-2774-4854-96b1-0a364bde127a admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:57.431589 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221d40f3-2774-4854-96b1-0a364bde127a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:57.433330 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221d40f3-2774-4854-96b1-0a364bde127a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:57.464753 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1082/2167] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:07:57 2022] POST /identity/v3/projects => generated 369 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:07:57.475443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d83308-fdc2-4c32-9778-3d2e016e84eb None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:57.494054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d83308-fdc2-4c32-9778-3d2e016e84eb None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:57.560277 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9457a6c2-a22d-45ca-a68d-fbb8b80a53dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:57.561030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457a6c2-a22d-45ca-a68d-fbb8b80a53dd admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:57.561320 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457a6c2-a22d-45ca-a68d-fbb8b80a53dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:57.561597 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457a6c2-a22d-45ca-a68d-fbb8b80a53dd admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:57.562240 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9457a6c2-a22d-45ca-a68d-fbb8b80a53dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:57.564306 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9457a6c2-a22d-45ca-a68d-fbb8b80a53dd admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:57.586765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1086/2168] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:07:57 2022] POST /identity/v3/users => generated 291 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:07:57.590855 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d40f3-2774-4854-96b1-0a364bde127a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:57.602286 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d40f3-2774-4854-96b1-0a364bde127a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:57.606857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec143223-e9f3-490e-964c-dcc9b2474018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:57.607457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec143223-e9f3-490e-964c-dcc9b2474018 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:57.607768 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec143223-e9f3-490e-964c-dcc9b2474018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:57.608042 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec143223-e9f3-490e-964c-dcc9b2474018 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:57.608682 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec143223-e9f3-490e-964c-dcc9b2474018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:57.610359 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec143223-e9f3-490e-964c-dcc9b2474018 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:57.622637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1083/2169] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:07:57.626795 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9457a6c2-a22d-45ca-a68d-fbb8b80a53dd admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:57.638164 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9457a6c2-a22d-45ca-a68d-fbb8b80a53dd admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:57.643078 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9069aff-7555-4add-8f6a-ac578add31dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:57.643761 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9069aff-7555-4add-8f6a-ac578add31dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:57.644144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9069aff-7555-4add-8f6a-ac578add31dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:57.644479 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9069aff-7555-4add-8f6a-ac578add31dd admin admin] PATH_INFO: `/v3/projects/3f75b5127c2a4d3e994f6fa2b25c5da0/users/adf420c0a4b646d4a8a94446831cf882/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:57.659290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9069aff-7555-4add-8f6a-ac578add31dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f75b5127c2a4d3e994f6fa2b25c5da0, user_id=adf420c0a4b646d4a8a94446831cf882, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:57.662091 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9069aff-7555-4add-8f6a-ac578add31dd admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:57.676211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1087/2170] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:07:57 2022] PUT /identity/v3/projects/3f75b5127c2a4d3e994f6fa2b25c5da0/users/adf420c0a4b646d4a8a94446831cf882/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:07:57.680303 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec143223-e9f3-490e-964c-dcc9b2474018 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:57.691743 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec143223-e9f3-490e-964c-dcc9b2474018 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:57.750136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0321bd5-c97b-4228-b77c-e5298f49ba2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:57.750787 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0321bd5-c97b-4228-b77c-e5298f49ba2a admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:57.751067 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0321bd5-c97b-4228-b77c-e5298f49ba2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:57.751348 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0321bd5-c97b-4228-b77c-e5298f49ba2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:57.752028 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0321bd5-c97b-4228-b77c-e5298f49ba2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:57.753763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0321bd5-c97b-4228-b77c-e5298f49ba2a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:57.764794 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1084/2171] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:07:57.769108 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9069aff-7555-4add-8f6a-ac578add31dd admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:57.774742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:07:57.781759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9069aff-7555-4add-8f6a-ac578add31dd admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:57.787380 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c71d29-66a9-423a-ab27-34eaab38bf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:57.788020 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c71d29-66a9-423a-ab27-34eaab38bf95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:57.788323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c71d29-66a9-423a-ab27-34eaab38bf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:57.788569 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c71d29-66a9-423a-ab27-34eaab38bf95 admin admin] PATH_INFO: `/v3/projects/3f75b5127c2a4d3e994f6fa2b25c5da0/users/adf420c0a4b646d4a8a94446831cf882/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:57.790601 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c71d29-66a9-423a-ab27-34eaab38bf95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f75b5127c2a4d3e994f6fa2b25c5da0, user_id=adf420c0a4b646d4a8a94446831cf882, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:57.793411 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c71d29-66a9-423a-ab27-34eaab38bf95 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:57.804690 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1088/2172] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:07:57 2022] PUT /identity/v3/projects/3f75b5127c2a4d3e994f6fa2b25c5da0/users/adf420c0a4b646d4a8a94446831cf882/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:07:57.814679 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553858c-9e3f-4717-a774-24b1ab49a761 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:57.814965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553858c-9e3f-4717-a774-24b1ab49a761 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:57.815236 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553858c-9e3f-4717-a774-24b1ab49a761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:57.842506 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f553858c-9e3f-4717-a774-24b1ab49a761 None None] MFA Rules not processed for user `adf420c0a4b646d4a8a94446831cf882`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:07:57.843746 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f553858c-9e3f-4717-a774-24b1ab49a761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:07:57.913098 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1085/2173] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:07:57 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:07:57.924629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c71d29-66a9-423a-ab27-34eaab38bf95 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:57.926951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b4c71d29-66a9-423a-ab27-34eaab38bf95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:07:57.983714 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c71d29-66a9-423a-ab27-34eaab38bf95 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:57.987290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4611af-6ea8-478d-a268-dfc58e42d7f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:57.989015 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4611af-6ea8-478d-a268-dfc58e42d7f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:57.989015 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4611af-6ea8-478d-a268-dfc58e42d7f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:57.989260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4611af-6ea8-478d-a268-dfc58e42d7f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:57.997999 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4611af-6ea8-478d-a268-dfc58e42d7f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:57.999429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4611af-6ea8-478d-a268-dfc58e42d7f1 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:58.008786 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1089/2174] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:07:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:07:58.054586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553858c-9e3f-4717-a774-24b1ab49a761 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:07:58.056042 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f553858c-9e3f-4717-a774-24b1ab49a761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:07:58.068149 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553858c-9e3f-4717-a774-24b1ab49a761 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:07:58.071741 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e6dac-6546-4d5e-af83-a26e57479112 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:07:58.072340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e6dac-6546-4d5e-af83-a26e57479112 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:07:58.072621 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e6dac-6546-4d5e-af83-a26e57479112 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:07:58.072894 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e6dac-6546-4d5e-af83-a26e57479112 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:07:58.106522 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:07:58.131750 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e6dac-6546-4d5e-af83-a26e57479112 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:07:58.133082 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e6dac-6546-4d5e-af83-a26e57479112 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:07:58.142158 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1086/2175] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:07:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:08:01.427280 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4611af-6ea8-478d-a268-dfc58e42d7f1 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:01.491710 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4611af-6ea8-478d-a268-dfc58e42d7f1 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:01.496256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a256164-7a4e-4a65-a717-f71e132021da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:01.497200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a256164-7a4e-4a65-a717-f71e132021da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:01.497455 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a256164-7a4e-4a65-a717-f71e132021da admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:01.497728 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a256164-7a4e-4a65-a717-f71e132021da admin admin] PATH_INFO: `/v3/users/697ec9fdb3ef40238bb285a931bc0bce` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:01.498849 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a256164-7a4e-4a65-a717-f71e132021da admin admin] RBAC: Authorizing `identity:delete_user(user_id=697ec9fdb3ef40238bb285a931bc0bce)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:01.501122 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a256164-7a4e-4a65-a717-f71e132021da admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:01.547158 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4a256164-7a4e-4a65-a717-f71e132021da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '697ec9fdb3ef40238bb285a931bc0bce'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:01.547834 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4a256164-7a4e-4a65-a717-f71e132021da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '697ec9fdb3ef40238bb285a931bc0bce'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:01.555176 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4a256164-7a4e-4a65-a717-f71e132021da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '697ec9fdb3ef40238bb285a931bc0bce'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:01.562090 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4a256164-7a4e-4a65-a717-f71e132021da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '697ec9fdb3ef40238bb285a931bc0bce'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:01.562709 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4a256164-7a4e-4a65-a717-f71e132021da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '697ec9fdb3ef40238bb285a931bc0bce'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:01.582143 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1090/2176] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:08:01 2022] DELETE /identity/v3/users/697ec9fdb3ef40238bb285a931bc0bce => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:08:01.586692 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f7abf-1d6c-40f2-a709-f67d021e152c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:01.587188 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f7abf-1d6c-40f2-a709-f67d021e152c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:01.587315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f7abf-1d6c-40f2-a709-f67d021e152c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:01.606035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-cc6f7abf-1d6c-40f2-a709-f67d021e152c None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:08:01.607233 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cc6f7abf-1d6c-40f2-a709-f67d021e152c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:01.667540 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1087/2177] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:08:01.677512 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a256164-7a4e-4a65-a717-f71e132021da admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:01.679029 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4a256164-7a4e-4a65-a717-f71e132021da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:01.692914 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:08:01.738968 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a256164-7a4e-4a65-a717-f71e132021da admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:01.742747 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726ab2f-fab3-460f-879d-03ee6cb1d499 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:01.743355 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726ab2f-fab3-460f-879d-03ee6cb1d499 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:01.743665 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726ab2f-fab3-460f-879d-03ee6cb1d499 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:01.743936 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726ab2f-fab3-460f-879d-03ee6cb1d499 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:01.752759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726ab2f-fab3-460f-879d-03ee6cb1d499 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:01.754701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726ab2f-fab3-460f-879d-03ee6cb1d499 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:01.764432 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1091/2178] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:08:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:08:02.030376 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f7abf-1d6c-40f2-a709-f67d021e152c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:02.031908 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cc6f7abf-1d6c-40f2-a709-f67d021e152c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:02.049858 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f7abf-1d6c-40f2-a709-f67d021e152c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:02.055223 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35f826-dad2-4bd3-9423-28b933d937c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:02.055539 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35f826-dad2-4bd3-9423-28b933d937c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:02.055849 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35f826-dad2-4bd3-9423-28b933d937c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:02.056566 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a35f826-dad2-4bd3-9423-28b933d937c4 admin admin] PATH_INFO: `/v3/projects/1ca30fa38e20438a9315f2d63634cf5a` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:02.057418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a35f826-dad2-4bd3-9423-28b933d937c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ca30fa38e20438a9315f2d63634cf5a)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:02.059369 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a35f826-dad2-4bd3-9423-28b933d937c4 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:02.107718 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7a35f826-dad2-4bd3-9423-28b933d937c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ca30fa38e20438a9315f2d63634cf5a', 'request_id': 'req-7a35f826-dad2-4bd3-9423-28b933d937c4'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:02.114855 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7a35f826-dad2-4bd3-9423-28b933d937c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ca30fa38e20438a9315f2d63634cf5a', 'request_id': 'req-7a35f826-dad2-4bd3-9423-28b933d937c4'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:02.120652 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7a35f826-dad2-4bd3-9423-28b933d937c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ca30fa38e20438a9315f2d63634cf5a', 'request_id': 'req-7a35f826-dad2-4bd3-9423-28b933d937c4'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:02.131790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7a35f826-dad2-4bd3-9423-28b933d937c4 admin admin] The token cache is being invalidate because project 1ca30fa38e20438a9315f2d63634cf5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:08:02.132082 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7a35f826-dad2-4bd3-9423-28b933d937c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:02.132756 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7a35f826-dad2-4bd3-9423-28b933d937c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:02.134270 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1088/2179] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:08:02 2022] DELETE /identity/v3/projects/1ca30fa38e20438a9315f2d63634cf5a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:08:02.138522 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726ab2f-fab3-460f-879d-03ee6cb1d499 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:02.149183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726ab2f-fab3-460f-879d-03ee6cb1d499 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:02.204868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc51cb-cae5-4dcf-9946-ff500ed15dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:02.205425 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc51cb-cae5-4dcf-9946-ff500ed15dff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:02.205705 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc51cb-cae5-4dcf-9946-ff500ed15dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:02.205978 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc51cb-cae5-4dcf-9946-ff500ed15dff admin admin] PATH_INFO: `/v3/users/adf420c0a4b646d4a8a94446831cf882` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:02.207648 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc51cb-cae5-4dcf-9946-ff500ed15dff admin admin] RBAC: Authorizing `identity:delete_user(user_id=adf420c0a4b646d4a8a94446831cf882)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:02.209665 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc51cb-cae5-4dcf-9946-ff500ed15dff admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:02.259475 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-93bc51cb-cae5-4dcf-9946-ff500ed15dff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adf420c0a4b646d4a8a94446831cf882'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:02.260158 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-93bc51cb-cae5-4dcf-9946-ff500ed15dff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adf420c0a4b646d4a8a94446831cf882'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:02.269876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-93bc51cb-cae5-4dcf-9946-ff500ed15dff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adf420c0a4b646d4a8a94446831cf882'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:02.276877 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-93bc51cb-cae5-4dcf-9946-ff500ed15dff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adf420c0a4b646d4a8a94446831cf882'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:02.277499 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-93bc51cb-cae5-4dcf-9946-ff500ed15dff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adf420c0a4b646d4a8a94446831cf882'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:02.294696 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1092/2180] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:08:02 2022] DELETE /identity/v3/users/adf420c0a4b646d4a8a94446831cf882 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:08:02.568131 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35f826-dad2-4bd3-9423-28b933d937c4 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:02.569975 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7a35f826-dad2-4bd3-9423-28b933d937c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:02.583592 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a35f826-dad2-4bd3-9423-28b933d937c4 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:02.637675 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:08:02.644948 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef0882-aabd-4e1d-bf0f-60ff0ea460ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:02.646105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef0882-aabd-4e1d-bf0f-60ff0ea460ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:02.646531 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef0882-aabd-4e1d-bf0f-60ff0ea460ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:02.646887 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef0882-aabd-4e1d-bf0f-60ff0ea460ea admin admin] PATH_INFO: `/v3/projects/3f75b5127c2a4d3e994f6fa2b25c5da0` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:02.648303 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef0882-aabd-4e1d-bf0f-60ff0ea460ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f75b5127c2a4d3e994f6fa2b25c5da0)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:02.650676 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef0882-aabd-4e1d-bf0f-60ff0ea460ea admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:02.699021 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-35ef0882-aabd-4e1d-bf0f-60ff0ea460ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f75b5127c2a4d3e994f6fa2b25c5da0', 'request_id': 'req-35ef0882-aabd-4e1d-bf0f-60ff0ea460ea'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:02.706600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-35ef0882-aabd-4e1d-bf0f-60ff0ea460ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f75b5127c2a4d3e994f6fa2b25c5da0', 'request_id': 'req-35ef0882-aabd-4e1d-bf0f-60ff0ea460ea'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:02.712165 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-35ef0882-aabd-4e1d-bf0f-60ff0ea460ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f75b5127c2a4d3e994f6fa2b25c5da0', 'request_id': 'req-35ef0882-aabd-4e1d-bf0f-60ff0ea460ea'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:02.727076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-35ef0882-aabd-4e1d-bf0f-60ff0ea460ea admin admin] The token cache is being invalidate because project 3f75b5127c2a4d3e994f6fa2b25c5da0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:08:02.727368 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-35ef0882-aabd-4e1d-bf0f-60ff0ea460ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:02.728058 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-35ef0882-aabd-4e1d-bf0f-60ff0ea460ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:02.729544 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1089/2181] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:08:02 2022] DELETE /identity/v3/projects/3f75b5127c2a4d3e994f6fa2b25c5da0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:08:02.734598 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1b3b3-879a-4837-8f58-35c4953d81cb None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:02.735082 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1b3b3-879a-4837-8f58-35c4953d81cb None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:02.735082 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1b3b3-879a-4837-8f58-35c4953d81cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:02.756242 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-90c1b3b3-879a-4837-8f58-35c4953d81cb None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:08:02.759252 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-90c1b3b3-879a-4837-8f58-35c4953d81cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:02.789512 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:08:02.820897 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1093/2182] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:08:02.829869 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f921fe-8afd-417d-b52c-531775224844 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:02.830156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f921fe-8afd-417d-b52c-531775224844 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:02.830431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f921fe-8afd-417d-b52c-531775224844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:02.851042 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b7f921fe-8afd-417d-b52c-531775224844 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:08:02.852274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b7f921fe-8afd-417d-b52c-531775224844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:02.860153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1090/2183] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:08:02.864514 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1b3b3-879a-4837-8f58-35c4953d81cb None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:02.874265 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1b3b3-879a-4837-8f58-35c4953d81cb None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:02.879044 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d14fd-d5ae-4e3f-8936-a26a472236d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:02.879663 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d14fd-d5ae-4e3f-8936-a26a472236d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:02.879946 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d14fd-d5ae-4e3f-8936-a26a472236d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:02.880220 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d14fd-d5ae-4e3f-8936-a26a472236d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:02.881716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d14fd-d5ae-4e3f-8936-a26a472236d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:08:02.882022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d14fd-d5ae-4e3f-8936-a26a472236d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:02.883651 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d14fd-d5ae-4e3f-8936-a26a472236d1 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:02.896258 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1094/2184] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:08:02.900472 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f921fe-8afd-417d-b52c-531775224844 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:02.910322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f921fe-8afd-417d-b52c-531775224844 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:02.914927 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db416f8-7f24-4421-b9d9-a579da3bca99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:02.915521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db416f8-7f24-4421-b9d9-a579da3bca99 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:02.915828 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db416f8-7f24-4421-b9d9-a579da3bca99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:02.916105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db416f8-7f24-4421-b9d9-a579da3bca99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:02.916754 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db416f8-7f24-4421-b9d9-a579da3bca99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:02.918721 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db416f8-7f24-4421-b9d9-a579da3bca99 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:02.945319 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1091/2185] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:08:02 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:08:02.949365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d14fd-d5ae-4e3f-8936-a26a472236d1 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:02.960204 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d14fd-d5ae-4e3f-8936-a26a472236d1 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:03.019651 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e721bf8-ca77-4cab-bcba-4e66e91768ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:03.020258 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e721bf8-ca77-4cab-bcba-4e66e91768ae admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:03.020540 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e721bf8-ca77-4cab-bcba-4e66e91768ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:03.020812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e721bf8-ca77-4cab-bcba-4e66e91768ae admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:03.021448 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e721bf8-ca77-4cab-bcba-4e66e91768ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:03.023384 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e721bf8-ca77-4cab-bcba-4e66e91768ae admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:03.044331 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1095/2186] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:08:02 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:08:03.048393 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db416f8-7f24-4421-b9d9-a579da3bca99 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:03.058881 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db416f8-7f24-4421-b9d9-a579da3bca99 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:03.063416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f267499-e92b-4e81-a36f-80612798b32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:03.064052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f267499-e92b-4e81-a36f-80612798b32d admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:03.064327 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f267499-e92b-4e81-a36f-80612798b32d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:03.064599 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f267499-e92b-4e81-a36f-80612798b32d admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:03.065236 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f267499-e92b-4e81-a36f-80612798b32d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:03.067778 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f267499-e92b-4e81-a36f-80612798b32d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:03.077602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1092/2187] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:08:03.082379 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e721bf8-ca77-4cab-bcba-4e66e91768ae admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:03.092555 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e721bf8-ca77-4cab-bcba-4e66e91768ae admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:03.097233 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01572ec4-620b-40a6-8be5-8d9071dcd636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:03.097825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01572ec4-620b-40a6-8be5-8d9071dcd636 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:03.098102 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01572ec4-620b-40a6-8be5-8d9071dcd636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:03.098376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01572ec4-620b-40a6-8be5-8d9071dcd636 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:03.099022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01572ec4-620b-40a6-8be5-8d9071dcd636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:03.100655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01572ec4-620b-40a6-8be5-8d9071dcd636 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:03.111704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1096/2188] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:08:03.116061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f267499-e92b-4e81-a36f-80612798b32d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:03.134653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f267499-e92b-4e81-a36f-80612798b32d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:03.134653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6d26e-526b-466d-b537-6c26e19906e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:03.134653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6d26e-526b-466d-b537-6c26e19906e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:03.134653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6d26e-526b-466d-b537-6c26e19906e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:03.135027 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6d26e-526b-466d-b537-6c26e19906e6 admin admin] PATH_INFO: `/v3/projects/266036e72a234d9baa79b39cc8398ee5/users/8fc187ec2b044907b6bd3cac7254254e/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:03.149532 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6d26e-526b-466d-b537-6c26e19906e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=266036e72a234d9baa79b39cc8398ee5, user_id=8fc187ec2b044907b6bd3cac7254254e, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:03.152264 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6d26e-526b-466d-b537-6c26e19906e6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:03.164991 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1093/2189] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:08:03 2022] PUT /identity/v3/projects/266036e72a234d9baa79b39cc8398ee5/users/8fc187ec2b044907b6bd3cac7254254e/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:08:03.175436 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef4080-7e0a-45e0-9b67-4e5c7da2dbb2 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:03.175840 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef4080-7e0a-45e0-9b67-4e5c7da2dbb2 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:03.176176 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef4080-7e0a-45e0-9b67-4e5c7da2dbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:03.203982 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-40ef4080-7e0a-45e0-9b67-4e5c7da2dbb2 None None] MFA Rules not processed for user `8fc187ec2b044907b6bd3cac7254254e`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:08:03.205196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-40ef4080-7e0a-45e0-9b67-4e5c7da2dbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:03.268344 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1097/2190] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:08:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:08:03.273468 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6d26e-526b-466d-b537-6c26e19906e6 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:03.335505 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6d26e-526b-466d-b537-6c26e19906e6 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:03.340121 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cfe326-897f-4521-93c7-7851ddb32b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:03.340725 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfe326-897f-4521-93c7-7851ddb32b6c admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:03.341009 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfe326-897f-4521-93c7-7851ddb32b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:03.341292 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfe326-897f-4521-93c7-7851ddb32b6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:03.341924 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cfe326-897f-4521-93c7-7851ddb32b6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:03.343856 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cfe326-897f-4521-93c7-7851ddb32b6c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:03.376369 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1094/2191] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:08:03 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:08:03.381042 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef4080-7e0a-45e0-9b67-4e5c7da2dbb2 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:03.391876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef4080-7e0a-45e0-9b67-4e5c7da2dbb2 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:03.424120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:08:03.454016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8817f-6561-4eb5-906f-0b562a23ca76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:03.454620 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8817f-6561-4eb5-906f-0b562a23ca76 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:03.454902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8817f-6561-4eb5-906f-0b562a23ca76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:03.455179 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8817f-6561-4eb5-906f-0b562a23ca76 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:03.455847 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8817f-6561-4eb5-906f-0b562a23ca76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:03.458367 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8817f-6561-4eb5-906f-0b562a23ca76 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:03.483021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1098/2192] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:08:03 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:08:03.484496 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cfe326-897f-4521-93c7-7851ddb32b6c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:03.495837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cfe326-897f-4521-93c7-7851ddb32b6c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:03.500443 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854e79d-2745-41c3-9dd0-2432ea5fbd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:03.501041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854e79d-2745-41c3-9dd0-2432ea5fbd86 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:03.501320 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854e79d-2745-41c3-9dd0-2432ea5fbd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:03.501595 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854e79d-2745-41c3-9dd0-2432ea5fbd86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:03.502255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854e79d-2745-41c3-9dd0-2432ea5fbd86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:03.503938 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854e79d-2745-41c3-9dd0-2432ea5fbd86 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:03.506028 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:08:03.516140 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1095/2193] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:08:03.520250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8817f-6561-4eb5-906f-0b562a23ca76 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:03.530162 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8817f-6561-4eb5-906f-0b562a23ca76 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:03.535462 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05f272-3040-40b6-86dc-dafe9386e74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:03.536081 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05f272-3040-40b6-86dc-dafe9386e74d admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:03.536362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05f272-3040-40b6-86dc-dafe9386e74d admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:03.536633 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05f272-3040-40b6-86dc-dafe9386e74d admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:03.537267 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05f272-3040-40b6-86dc-dafe9386e74d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:03.538862 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05f272-3040-40b6-86dc-dafe9386e74d admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:03.548589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1099/2194] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:08:03.552579 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854e79d-2745-41c3-9dd0-2432ea5fbd86 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:03.562640 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854e79d-2745-41c3-9dd0-2432ea5fbd86 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:03.567520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9567a31-49ec-4f96-9133-7b59b9df1825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:03.568204 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9567a31-49ec-4f96-9133-7b59b9df1825 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:03.568494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9567a31-49ec-4f96-9133-7b59b9df1825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:03.568771 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9567a31-49ec-4f96-9133-7b59b9df1825 admin admin] PATH_INFO: `/v3/projects/06e843b684754dd9b863fa2f7ed28b29/users/f53ff2943e8b4ef0bfc4b3a45572e0b0/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:03.583789 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9567a31-49ec-4f96-9133-7b59b9df1825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06e843b684754dd9b863fa2f7ed28b29, user_id=f53ff2943e8b4ef0bfc4b3a45572e0b0, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:03.586457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9567a31-49ec-4f96-9133-7b59b9df1825 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:03.599575 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1096/2195] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:08:03 2022] PUT /identity/v3/projects/06e843b684754dd9b863fa2f7ed28b29/users/f53ff2943e8b4ef0bfc4b3a45572e0b0/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:08:03.609635 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd131f2b-cf35-4c66-b3ce-80af8a2b1076 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:03.609958 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd131f2b-cf35-4c66-b3ce-80af8a2b1076 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:03.610268 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd131f2b-cf35-4c66-b3ce-80af8a2b1076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:03.639665 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-dd131f2b-cf35-4c66-b3ce-80af8a2b1076 None None] MFA Rules not processed for user `f53ff2943e8b4ef0bfc4b3a45572e0b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:08:03.640926 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-dd131f2b-cf35-4c66-b3ce-80af8a2b1076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:03.704435 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1100/2196] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:08:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:08:03.709536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9567a31-49ec-4f96-9133-7b59b9df1825 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:03.772150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9567a31-49ec-4f96-9133-7b59b9df1825 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:03.777029 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf2f40-2617-4af8-94bf-02796c2de9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:03.777638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf2f40-2617-4af8-94bf-02796c2de9b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:03.777919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf2f40-2617-4af8-94bf-02796c2de9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:03.778193 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf2f40-2617-4af8-94bf-02796c2de9b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:03.778811 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf2f40-2617-4af8-94bf-02796c2de9b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:03.780747 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf2f40-2617-4af8-94bf-02796c2de9b7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:03.808613 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1097/2197] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:08:03 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:08:03.812676 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd131f2b-cf35-4c66-b3ce-80af8a2b1076 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:03.824351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd131f2b-cf35-4c66-b3ce-80af8a2b1076 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:03.881373 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a589f53-9c53-4edc-b06f-13fe27e2f354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:03.881975 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a589f53-9c53-4edc-b06f-13fe27e2f354 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:03.882257 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a589f53-9c53-4edc-b06f-13fe27e2f354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:03.882533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a589f53-9c53-4edc-b06f-13fe27e2f354 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:03.883171 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a589f53-9c53-4edc-b06f-13fe27e2f354 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:03.885130 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a589f53-9c53-4edc-b06f-13fe27e2f354 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:03.906553 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1101/2198] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:08:03 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:08:03.910489 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf2f40-2617-4af8-94bf-02796c2de9b7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:03.920687 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf2f40-2617-4af8-94bf-02796c2de9b7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:03.925550 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338017c-211d-4e15-8968-92429024e593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:03.926148 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338017c-211d-4e15-8968-92429024e593 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:03.926432 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338017c-211d-4e15-8968-92429024e593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:03.926710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338017c-211d-4e15-8968-92429024e593 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:03.927355 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338017c-211d-4e15-8968-92429024e593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:03.929286 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338017c-211d-4e15-8968-92429024e593 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:03.939767 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1098/2199] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:08:03.944478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a589f53-9c53-4edc-b06f-13fe27e2f354 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:03.954145 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a589f53-9c53-4edc-b06f-13fe27e2f354 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:03.959432 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9c396-098e-43aa-9698-f8054b937019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:03.960158 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9c396-098e-43aa-9698-f8054b937019 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:03.960437 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9c396-098e-43aa-9698-f8054b937019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:03.960706 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9c396-098e-43aa-9698-f8054b937019 admin admin] PATH_INFO: `/v3/projects/13cd37c4b2154dcbbcb707d4afc565b7/users/85b93a9c7a5e432fa3f556d038622978/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:03.974958 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9c396-098e-43aa-9698-f8054b937019 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13cd37c4b2154dcbbcb707d4afc565b7, user_id=85b93a9c7a5e432fa3f556d038622978, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:03.977694 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9c396-098e-43aa-9698-f8054b937019 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:03.990267 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1102/2200] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:08:03 2022] PUT /identity/v3/projects/13cd37c4b2154dcbbcb707d4afc565b7/users/85b93a9c7a5e432fa3f556d038622978/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:08:03.994161 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338017c-211d-4e15-8968-92429024e593 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:04.005097 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338017c-211d-4e15-8968-92429024e593 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:04.060822 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edda66f-cf90-4d16-b62e-00ae0d3709f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:04.061361 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edda66f-cf90-4d16-b62e-00ae0d3709f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:04.061642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edda66f-cf90-4d16-b62e-00ae0d3709f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:04.061917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edda66f-cf90-4d16-b62e-00ae0d3709f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:04.062556 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edda66f-cf90-4d16-b62e-00ae0d3709f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:04.064187 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edda66f-cf90-4d16-b62e-00ae0d3709f8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:04.074383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1099/2201] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:08:04.078537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9c396-098e-43aa-9698-f8054b937019 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:04.084551 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:08:04.090529 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9c396-098e-43aa-9698-f8054b937019 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:04.095308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd20982-dc52-4445-8cc0-c23e2f90511d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:04.096010 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd20982-dc52-4445-8cc0-c23e2f90511d admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:04.096291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd20982-dc52-4445-8cc0-c23e2f90511d admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:04.096588 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd20982-dc52-4445-8cc0-c23e2f90511d admin admin] PATH_INFO: `/v3/projects/13cd37c4b2154dcbbcb707d4afc565b7/users/85b93a9c7a5e432fa3f556d038622978/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:04.098524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd20982-dc52-4445-8cc0-c23e2f90511d admin admin] RBAC: Authorizing `identity:create_grant(project_id=13cd37c4b2154dcbbcb707d4afc565b7, user_id=85b93a9c7a5e432fa3f556d038622978, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:04.101197 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd20982-dc52-4445-8cc0-c23e2f90511d admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:04.111180 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1103/2202] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:08:04 2022] PUT /identity/v3/projects/13cd37c4b2154dcbbcb707d4afc565b7/users/85b93a9c7a5e432fa3f556d038622978/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:08:04.122814 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfb508-c649-4140-aed6-c455eee3a5a7 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:04.123109 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfb508-c649-4140-aed6-c455eee3a5a7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:04.123390 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfb508-c649-4140-aed6-c455eee3a5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:04.152495 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-38bfb508-c649-4140-aed6-c455eee3a5a7 None None] MFA Rules not processed for user `85b93a9c7a5e432fa3f556d038622978`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:08:04.153714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-38bfb508-c649-4140-aed6-c455eee3a5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:04.192840 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:08:04.229153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1100/2203] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:08:04 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:08:04.240746 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd20982-dc52-4445-8cc0-c23e2f90511d admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:04.242464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9dd20982-dc52-4445-8cc0-c23e2f90511d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:04.301321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd20982-dc52-4445-8cc0-c23e2f90511d admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:04.304939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe99811-41b7-4ac3-ad71-55b248cb89a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:04.305534 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe99811-41b7-4ac3-ad71-55b248cb89a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:04.305814 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe99811-41b7-4ac3-ad71-55b248cb89a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:04.306088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe99811-41b7-4ac3-ad71-55b248cb89a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:04.314848 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe99811-41b7-4ac3-ad71-55b248cb89a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:04.317173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe99811-41b7-4ac3-ad71-55b248cb89a2 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:04.326802 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1104/2204] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:08:04 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:08:04.827459 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bfb508-c649-4140-aed6-c455eee3a5a7 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:04.828901 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-38bfb508-c649-4140-aed6-c455eee3a5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:04.841277 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bfb508-c649-4140-aed6-c455eee3a5a7 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:04.844887 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76791555-9fc7-46ca-9a80-ef5b326eebd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:04.845489 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76791555-9fc7-46ca-9a80-ef5b326eebd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:04.845775 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76791555-9fc7-46ca-9a80-ef5b326eebd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:04.846049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76791555-9fc7-46ca-9a80-ef5b326eebd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:04.904993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76791555-9fc7-46ca-9a80-ef5b326eebd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:04.906560 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76791555-9fc7-46ca-9a80-ef5b326eebd6 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:04.915895 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1101/2205] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:08:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:08:11.056562 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe99811-41b7-4ac3-ad71-55b248cb89a2 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:11.065315 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe99811-41b7-4ac3-ad71-55b248cb89a2 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:11.069440 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435e193-829c-4eca-8f9f-f71655143fa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:11.070120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435e193-829c-4eca-8f9f-f71655143fa7 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:11.070467 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435e193-829c-4eca-8f9f-f71655143fa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:11.070806 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435e193-829c-4eca-8f9f-f71655143fa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:11.128308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435e193-829c-4eca-8f9f-f71655143fa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:11.129824 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435e193-829c-4eca-8f9f-f71655143fa7 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:11.138928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1105/2206] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:08:11 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:08:13.317213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76791555-9fc7-46ca-9a80-ef5b326eebd6 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:13.380377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76791555-9fc7-46ca-9a80-ef5b326eebd6 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:13.385008 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca06aa4-389e-4e71-a796-13de7a040b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:13.385631 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca06aa4-389e-4e71-a796-13de7a040b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:13.385907 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca06aa4-389e-4e71-a796-13de7a040b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:13.386179 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca06aa4-389e-4e71-a796-13de7a040b1b admin admin] PATH_INFO: `/v3/users/8fc187ec2b044907b6bd3cac7254254e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:13.387314 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca06aa4-389e-4e71-a796-13de7a040b1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fc187ec2b044907b6bd3cac7254254e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:13.389509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca06aa4-389e-4e71-a796-13de7a040b1b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:13.435462 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bca06aa4-389e-4e71-a796-13de7a040b1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fc187ec2b044907b6bd3cac7254254e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:13.436200 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bca06aa4-389e-4e71-a796-13de7a040b1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fc187ec2b044907b6bd3cac7254254e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:13.443961 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bca06aa4-389e-4e71-a796-13de7a040b1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fc187ec2b044907b6bd3cac7254254e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:13.444571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bca06aa4-389e-4e71-a796-13de7a040b1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fc187ec2b044907b6bd3cac7254254e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:13.452032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-bca06aa4-389e-4e71-a796-13de7a040b1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fc187ec2b044907b6bd3cac7254254e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:13.457864 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:08:13.472000 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1102/2207] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:08:13 2022] DELETE /identity/v3/users/8fc187ec2b044907b6bd3cac7254254e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:08:13.476660 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cc8fb-e712-49ac-8ba8-21833ee9d5f7 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:13.477122 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cc8fb-e712-49ac-8ba8-21833ee9d5f7 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:13.477337 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cc8fb-e712-49ac-8ba8-21833ee9d5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:13.497760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-f43cc8fb-e712-49ac-8ba8-21833ee9d5f7 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:08:13.499007 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f43cc8fb-e712-49ac-8ba8-21833ee9d5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:13.557725 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1106/2208] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:08:13.568776 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca06aa4-389e-4e71-a796-13de7a040b1b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:13.570200 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bca06aa4-389e-4e71-a796-13de7a040b1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:13.626762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca06aa4-389e-4e71-a796-13de7a040b1b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:13.631111 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10141ea2-ed84-45b2-9ea2-607a3237cf8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:13.631736 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10141ea2-ed84-45b2-9ea2-607a3237cf8f service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:13.632015 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10141ea2-ed84-45b2-9ea2-607a3237cf8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:13.632282 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10141ea2-ed84-45b2-9ea2-607a3237cf8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:13.641376 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10141ea2-ed84-45b2-9ea2-607a3237cf8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:13.642801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10141ea2-ed84-45b2-9ea2-607a3237cf8f service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:13.652389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1103/2209] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:08:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:08:13.913232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cc8fb-e712-49ac-8ba8-21833ee9d5f7 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:13.915322 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f43cc8fb-e712-49ac-8ba8-21833ee9d5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:13.928162 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cc8fb-e712-49ac-8ba8-21833ee9d5f7 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:13.933358 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65632120-5913-4d45-bdf2-5e5dcd5841aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:13.933993 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65632120-5913-4d45-bdf2-5e5dcd5841aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:13.934676 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65632120-5913-4d45-bdf2-5e5dcd5841aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:13.934676 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65632120-5913-4d45-bdf2-5e5dcd5841aa admin admin] PATH_INFO: `/v3/projects/266036e72a234d9baa79b39cc8398ee5` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:13.935696 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65632120-5913-4d45-bdf2-5e5dcd5841aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=266036e72a234d9baa79b39cc8398ee5)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:13.937661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65632120-5913-4d45-bdf2-5e5dcd5841aa admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:13.968911 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:08:13.988088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-65632120-5913-4d45-bdf2-5e5dcd5841aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '266036e72a234d9baa79b39cc8398ee5', 'request_id': 'req-65632120-5913-4d45-bdf2-5e5dcd5841aa'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:13.995382 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-65632120-5913-4d45-bdf2-5e5dcd5841aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '266036e72a234d9baa79b39cc8398ee5', 'request_id': 'req-65632120-5913-4d45-bdf2-5e5dcd5841aa'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:14.000988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-65632120-5913-4d45-bdf2-5e5dcd5841aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '266036e72a234d9baa79b39cc8398ee5', 'request_id': 'req-65632120-5913-4d45-bdf2-5e5dcd5841aa'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:14.012851 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-65632120-5913-4d45-bdf2-5e5dcd5841aa admin admin] The token cache is being invalidate because project 266036e72a234d9baa79b39cc8398ee5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:08:14.013139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-65632120-5913-4d45-bdf2-5e5dcd5841aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:14.013780 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-65632120-5913-4d45-bdf2-5e5dcd5841aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:14.015230 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1107/2210] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:08:13 2022] DELETE /identity/v3/projects/266036e72a234d9baa79b39cc8398ee5 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:08:14.019273 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10141ea2-ed84-45b2-9ea2-607a3237cf8f service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:14.029187 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10141ea2-ed84-45b2-9ea2-607a3237cf8f service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:14.087975 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505c71c-2b92-4600-b080-97fe0d14da2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:14.088616 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505c71c-2b92-4600-b080-97fe0d14da2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:14.088902 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505c71c-2b92-4600-b080-97fe0d14da2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:14.089181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505c71c-2b92-4600-b080-97fe0d14da2f admin admin] PATH_INFO: `/v3/users/f53ff2943e8b4ef0bfc4b3a45572e0b0` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:14.090309 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505c71c-2b92-4600-b080-97fe0d14da2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f53ff2943e8b4ef0bfc4b3a45572e0b0)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:14.092327 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505c71c-2b92-4600-b080-97fe0d14da2f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:14.138379 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e505c71c-2b92-4600-b080-97fe0d14da2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f53ff2943e8b4ef0bfc4b3a45572e0b0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:14.139011 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e505c71c-2b92-4600-b080-97fe0d14da2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f53ff2943e8b4ef0bfc4b3a45572e0b0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:14.146380 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e505c71c-2b92-4600-b080-97fe0d14da2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f53ff2943e8b4ef0bfc4b3a45572e0b0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:14.147117 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e505c71c-2b92-4600-b080-97fe0d14da2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f53ff2943e8b4ef0bfc4b3a45572e0b0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:14.154254 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e505c71c-2b92-4600-b080-97fe0d14da2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f53ff2943e8b4ef0bfc4b3a45572e0b0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:14.171933 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1104/2211] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:08:14 2022] DELETE /identity/v3/users/f53ff2943e8b4ef0bfc4b3a45572e0b0 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:08:14.444038 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65632120-5913-4d45-bdf2-5e5dcd5841aa admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:14.445562 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-65632120-5913-4d45-bdf2-5e5dcd5841aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:14.459527 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65632120-5913-4d45-bdf2-5e5dcd5841aa admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:14.518953 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe89f9-10e0-410b-9ec9-cd0d16be9be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:14.519626 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe89f9-10e0-410b-9ec9-cd0d16be9be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:14.519783 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe89f9-10e0-410b-9ec9-cd0d16be9be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:14.520059 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe89f9-10e0-410b-9ec9-cd0d16be9be5 admin admin] PATH_INFO: `/v3/projects/06e843b684754dd9b863fa2f7ed28b29` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:14.521198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbe89f9-10e0-410b-9ec9-cd0d16be9be5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06e843b684754dd9b863fa2f7ed28b29)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:14.523143 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbe89f9-10e0-410b-9ec9-cd0d16be9be5 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:14.569000 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6cbe89f9-10e0-410b-9ec9-cd0d16be9be5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06e843b684754dd9b863fa2f7ed28b29', 'request_id': 'req-6cbe89f9-10e0-410b-9ec9-cd0d16be9be5'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:14.576862 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6cbe89f9-10e0-410b-9ec9-cd0d16be9be5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06e843b684754dd9b863fa2f7ed28b29', 'request_id': 'req-6cbe89f9-10e0-410b-9ec9-cd0d16be9be5'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:14.582592 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6cbe89f9-10e0-410b-9ec9-cd0d16be9be5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06e843b684754dd9b863fa2f7ed28b29', 'request_id': 'req-6cbe89f9-10e0-410b-9ec9-cd0d16be9be5'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:14.593427 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6cbe89f9-10e0-410b-9ec9-cd0d16be9be5 admin admin] The token cache is being invalidate because project 06e843b684754dd9b863fa2f7ed28b29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:08:14.593715 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6cbe89f9-10e0-410b-9ec9-cd0d16be9be5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:14.594383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6cbe89f9-10e0-410b-9ec9-cd0d16be9be5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:14.596547 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1108/2212] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:08:14 2022] DELETE /identity/v3/projects/06e843b684754dd9b863fa2f7ed28b29 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:08:14.600792 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505c71c-2b92-4600-b080-97fe0d14da2f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:14.602207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e505c71c-2b92-4600-b080-97fe0d14da2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:14.615572 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505c71c-2b92-4600-b080-97fe0d14da2f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:14.620338 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:08:14.673173 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f1820-e334-491a-9920-fb5e2126218e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:14.673810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f1820-e334-491a-9920-fb5e2126218e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:14.674097 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f1820-e334-491a-9920-fb5e2126218e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:14.674941 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f1820-e334-491a-9920-fb5e2126218e admin admin] PATH_INFO: `/v3/users/85b93a9c7a5e432fa3f556d038622978` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:14.676094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f1820-e334-491a-9920-fb5e2126218e admin admin] RBAC: Authorizing `identity:delete_user(user_id=85b93a9c7a5e432fa3f556d038622978)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:14.678040 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f1820-e334-491a-9920-fb5e2126218e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:14.724733 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-448f1820-e334-491a-9920-fb5e2126218e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85b93a9c7a5e432fa3f556d038622978'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:14.725364 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-448f1820-e334-491a-9920-fb5e2126218e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85b93a9c7a5e432fa3f556d038622978'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:14.732464 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-448f1820-e334-491a-9920-fb5e2126218e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85b93a9c7a5e432fa3f556d038622978'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:14.733077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-448f1820-e334-491a-9920-fb5e2126218e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85b93a9c7a5e432fa3f556d038622978'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:14.740936 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-448f1820-e334-491a-9920-fb5e2126218e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85b93a9c7a5e432fa3f556d038622978'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:14.758064 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1105/2213] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:08:14 2022] DELETE /identity/v3/users/85b93a9c7a5e432fa3f556d038622978 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:08:15.029091 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe89f9-10e0-410b-9ec9-cd0d16be9be5 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:15.030530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6cbe89f9-10e0-410b-9ec9-cd0d16be9be5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:15.043704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe89f9-10e0-410b-9ec9-cd0d16be9be5 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:15.086717 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:08:15.104866 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae9a6a-3536-4a48-aa2c-22ff69620754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:15.105500 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae9a6a-3536-4a48-aa2c-22ff69620754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:15.105783 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae9a6a-3536-4a48-aa2c-22ff69620754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:15.106053 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae9a6a-3536-4a48-aa2c-22ff69620754 admin admin] PATH_INFO: `/v3/projects/13cd37c4b2154dcbbcb707d4afc565b7` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:15.107202 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae9a6a-3536-4a48-aa2c-22ff69620754 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13cd37c4b2154dcbbcb707d4afc565b7)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:15.109363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae9a6a-3536-4a48-aa2c-22ff69620754 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:15.156646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f0ae9a6a-3536-4a48-aa2c-22ff69620754 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13cd37c4b2154dcbbcb707d4afc565b7', 'request_id': 'req-f0ae9a6a-3536-4a48-aa2c-22ff69620754'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:15.164570 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f0ae9a6a-3536-4a48-aa2c-22ff69620754 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13cd37c4b2154dcbbcb707d4afc565b7', 'request_id': 'req-f0ae9a6a-3536-4a48-aa2c-22ff69620754'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:15.170264 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f0ae9a6a-3536-4a48-aa2c-22ff69620754 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13cd37c4b2154dcbbcb707d4afc565b7', 'request_id': 'req-f0ae9a6a-3536-4a48-aa2c-22ff69620754'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:15.181600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f0ae9a6a-3536-4a48-aa2c-22ff69620754 admin admin] The token cache is being invalidate because project 13cd37c4b2154dcbbcb707d4afc565b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:08:15.181883 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f0ae9a6a-3536-4a48-aa2c-22ff69620754 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:15.182516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f0ae9a6a-3536-4a48-aa2c-22ff69620754 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:15.184372 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1109/2214] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:08:15 2022] DELETE /identity/v3/projects/13cd37c4b2154dcbbcb707d4afc565b7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:08:15.190427 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc160d5-b833-43c4-beaf-c78a7a9d572e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:15.190934 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc160d5-b833-43c4-beaf-c78a7a9d572e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:15.191218 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc160d5-b833-43c4-beaf-c78a7a9d572e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:15.212396 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-edc160d5-b833-43c4-beaf-c78a7a9d572e None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:08:15.213629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-edc160d5-b833-43c4-beaf-c78a7a9d572e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:15.272145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1106/2215] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:08:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:08:15.281001 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb7169-f37a-4652-a9a6-47bc96d7bf77 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:15.281001 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb7169-f37a-4652-a9a6-47bc96d7bf77 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:15.281231 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb7169-f37a-4652-a9a6-47bc96d7bf77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:15.301879 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-f5fb7169-f37a-4652-a9a6-47bc96d7bf77 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:08:15.303102 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f5fb7169-f37a-4652-a9a6-47bc96d7bf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:15.311164 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1110/2216] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:08:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:08:15.315511 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc160d5-b833-43c4-beaf-c78a7a9d572e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:15.325239 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc160d5-b833-43c4-beaf-c78a7a9d572e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:15.329978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93385c-3c9f-4f42-b0f4-a5c3bb08d746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:15.331363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93385c-3c9f-4f42-b0f4-a5c3bb08d746 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:15.331683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93385c-3c9f-4f42-b0f4-a5c3bb08d746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:15.331964 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93385c-3c9f-4f42-b0f4-a5c3bb08d746 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:15.332621 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b93385c-3c9f-4f42-b0f4-a5c3bb08d746 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:08:15.332952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b93385c-3c9f-4f42-b0f4-a5c3bb08d746 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:15.334600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b93385c-3c9f-4f42-b0f4-a5c3bb08d746 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:15.346879 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1107/2217] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:08:15.350816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb7169-f37a-4652-a9a6-47bc96d7bf77 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:15.360093 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb7169-f37a-4652-a9a6-47bc96d7bf77 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:15.364810 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a456e2c-5468-4583-a1fb-2834f9b1df73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:15.365411 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a456e2c-5468-4583-a1fb-2834f9b1df73 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:15.365694 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a456e2c-5468-4583-a1fb-2834f9b1df73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:15.365970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a456e2c-5468-4583-a1fb-2834f9b1df73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:15.366609 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a456e2c-5468-4583-a1fb-2834f9b1df73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:15.368579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a456e2c-5468-4583-a1fb-2834f9b1df73 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:15.396740 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1111/2218] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:08:15 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:08:15.400835 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93385c-3c9f-4f42-b0f4-a5c3bb08d746 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:15.411034 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93385c-3c9f-4f42-b0f4-a5c3bb08d746 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:15.422179 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:08:15.469555 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008284e-3d35-41f9-ba5b-782d4abda189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:15.470356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008284e-3d35-41f9-ba5b-782d4abda189 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:15.470638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008284e-3d35-41f9-ba5b-782d4abda189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:15.470909 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008284e-3d35-41f9-ba5b-782d4abda189 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:15.471966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1008284e-3d35-41f9-ba5b-782d4abda189 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:15.473926 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1008284e-3d35-41f9-ba5b-782d4abda189 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:15.494701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1108/2219] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:08:15 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:08:15.498963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a456e2c-5468-4583-a1fb-2834f9b1df73 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:15.509856 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a456e2c-5468-4583-a1fb-2834f9b1df73 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:15.514719 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7facc1e2-194a-4bd3-945c-26443fe829d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:15.515323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facc1e2-194a-4bd3-945c-26443fe829d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:15.515631 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facc1e2-194a-4bd3-945c-26443fe829d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:15.515913 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facc1e2-194a-4bd3-945c-26443fe829d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:15.516566 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7facc1e2-194a-4bd3-945c-26443fe829d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:15.518841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7facc1e2-194a-4bd3-945c-26443fe829d0 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:15.528601 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1112/2220] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:08:15.532520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008284e-3d35-41f9-ba5b-782d4abda189 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:15.542714 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008284e-3d35-41f9-ba5b-782d4abda189 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:15.547395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782e1bb-ba1e-4b30-b8c0-ff771df0303c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:15.548042 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782e1bb-ba1e-4b30-b8c0-ff771df0303c admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:15.548322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782e1bb-ba1e-4b30-b8c0-ff771df0303c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:15.548640 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782e1bb-ba1e-4b30-b8c0-ff771df0303c admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:15.549318 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782e1bb-ba1e-4b30-b8c0-ff771df0303c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:15.551021 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782e1bb-ba1e-4b30-b8c0-ff771df0303c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:15.561606 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1109/2221] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:08:15.566295 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7facc1e2-194a-4bd3-945c-26443fe829d0 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:15.576165 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7facc1e2-194a-4bd3-945c-26443fe829d0 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:15.581462 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fca94-4109-4d0f-be98-e508f948f55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:15.582778 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fca94-4109-4d0f-be98-e508f948f55c admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:15.583264 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fca94-4109-4d0f-be98-e508f948f55c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:15.583544 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fca94-4109-4d0f-be98-e508f948f55c admin admin] PATH_INFO: `/v3/projects/b70be8c825ad4f8e93295a5b249a289d/users/3dd372a44c6a47baa736d165a67dd495/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:15.598211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398fca94-4109-4d0f-be98-e508f948f55c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b70be8c825ad4f8e93295a5b249a289d, user_id=3dd372a44c6a47baa736d165a67dd495, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:15.600937 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398fca94-4109-4d0f-be98-e508f948f55c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:15.613613 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1113/2222] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:08:15 2022] PUT /identity/v3/projects/b70be8c825ad4f8e93295a5b249a289d/users/3dd372a44c6a47baa736d165a67dd495/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:08:15.625837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d678171-c12c-487f-a8d5-7aa9bf7c48d0 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:15.626318 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d678171-c12c-487f-a8d5-7aa9bf7c48d0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:15.626419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d678171-c12c-487f-a8d5-7aa9bf7c48d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:15.654850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-1d678171-c12c-487f-a8d5-7aa9bf7c48d0 None None] MFA Rules not processed for user `3dd372a44c6a47baa736d165a67dd495`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:08:15.656096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1d678171-c12c-487f-a8d5-7aa9bf7c48d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:15.721312 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1110/2223] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:08:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:08:15.732465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fca94-4109-4d0f-be98-e508f948f55c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:15.734143 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-398fca94-4109-4d0f-be98-e508f948f55c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:15.791606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398fca94-4109-4d0f-be98-e508f948f55c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:15.795099 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf4874-e954-41ca-abe0-e5944884bdc2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89ff357209194276bc079d0c05898747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:15.795732 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf4874-e954-41ca-abe0-e5944884bdc2 service glance] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:15.796025 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf4874-e954-41ca-abe0-e5944884bdc2 service glance] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:15.796299 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf4874-e954-41ca-abe0-e5944884bdc2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:15.804959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaf4874-e954-41ca-abe0-e5944884bdc2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:15.806387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaf4874-e954-41ca-abe0-e5944884bdc2 service glance] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:15.815654 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1114/2224] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:08:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:08:25.718345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=59,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:08:25.811093 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:08:42.887806 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d678171-c12c-487f-a8d5-7aa9bf7c48d0 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:42.889218 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1d678171-c12c-487f-a8d5-7aa9bf7c48d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:42.952475 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d678171-c12c-487f-a8d5-7aa9bf7c48d0 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:42.957549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d274169-99c6-4dd1-9574-5af1976b94c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:42.958178 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d274169-99c6-4dd1-9574-5af1976b94c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:42.958464 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d274169-99c6-4dd1-9574-5af1976b94c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:42.958737 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d274169-99c6-4dd1-9574-5af1976b94c0 admin admin] PATH_INFO: `/v3/users/35611aeb415a416799fbb2cd4a080afe` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:42.959886 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d274169-99c6-4dd1-9574-5af1976b94c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35611aeb415a416799fbb2cd4a080afe)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:42.961804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d274169-99c6-4dd1-9574-5af1976b94c0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:43.008975 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4d274169-99c6-4dd1-9574-5af1976b94c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35611aeb415a416799fbb2cd4a080afe'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:43.009648 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4d274169-99c6-4dd1-9574-5af1976b94c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35611aeb415a416799fbb2cd4a080afe'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:43.017506 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4d274169-99c6-4dd1-9574-5af1976b94c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35611aeb415a416799fbb2cd4a080afe'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:43.019096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4d274169-99c6-4dd1-9574-5af1976b94c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35611aeb415a416799fbb2cd4a080afe'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:43.027106 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4d274169-99c6-4dd1-9574-5af1976b94c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35611aeb415a416799fbb2cd4a080afe'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:43.047298 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1111/2225] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:08:42 2022] DELETE /identity/v3/users/35611aeb415a416799fbb2cd4a080afe => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:08:43.052734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b990f-5aa9-4ca6-8aa9-1b7d3fdd3da5 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:43.053277 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b990f-5aa9-4ca6-8aa9-1b7d3fdd3da5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:43.053423 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b990f-5aa9-4ca6-8aa9-1b7d3fdd3da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:43.074519 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-106b990f-5aa9-4ca6-8aa9-1b7d3fdd3da5 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:08:43.075746 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-106b990f-5aa9-4ca6-8aa9-1b7d3fdd3da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:43.135657 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1115/2226] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:08:43.145981 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d274169-99c6-4dd1-9574-5af1976b94c0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:43.147382 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4d274169-99c6-4dd1-9574-5af1976b94c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:43.206116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d274169-99c6-4dd1-9574-5af1976b94c0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:43.209806 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eef4be-6869-4ac2-b67f-dabfc9dc0b86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:43.210401 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eef4be-6869-4ac2-b67f-dabfc9dc0b86 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:43.210678 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eef4be-6869-4ac2-b67f-dabfc9dc0b86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:43.210947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eef4be-6869-4ac2-b67f-dabfc9dc0b86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:43.219875 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eef4be-6869-4ac2-b67f-dabfc9dc0b86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:43.221472 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eef4be-6869-4ac2-b67f-dabfc9dc0b86 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:43.230937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1112/2227] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:08:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:08:43.495665 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b990f-5aa9-4ca6-8aa9-1b7d3fdd3da5 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:43.497095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-106b990f-5aa9-4ca6-8aa9-1b7d3fdd3da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:43.510064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b990f-5aa9-4ca6-8aa9-1b7d3fdd3da5 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:43.515336 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4781c3-a653-4916-816f-2add8619f5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:43.516038 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4781c3-a653-4916-816f-2add8619f5bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:43.516325 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4781c3-a653-4916-816f-2add8619f5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:43.516605 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4781c3-a653-4916-816f-2add8619f5bc admin admin] PATH_INFO: `/v3/projects/0b5606e8f3884720be78850662d9647f` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:43.517748 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4781c3-a653-4916-816f-2add8619f5bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b5606e8f3884720be78850662d9647f)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:43.520023 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4781c3-a653-4916-816f-2add8619f5bc admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:43.568358 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-fe4781c3-a653-4916-816f-2add8619f5bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b5606e8f3884720be78850662d9647f', 'request_id': 'req-fe4781c3-a653-4916-816f-2add8619f5bc'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:43.575599 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-fe4781c3-a653-4916-816f-2add8619f5bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b5606e8f3884720be78850662d9647f', 'request_id': 'req-fe4781c3-a653-4916-816f-2add8619f5bc'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:43.581462 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-fe4781c3-a653-4916-816f-2add8619f5bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b5606e8f3884720be78850662d9647f', 'request_id': 'req-fe4781c3-a653-4916-816f-2add8619f5bc'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:43.592547 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-fe4781c3-a653-4916-816f-2add8619f5bc admin admin] The token cache is being invalidate because project 0b5606e8f3884720be78850662d9647f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:08:43.592835 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-fe4781c3-a653-4916-816f-2add8619f5bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:43.593467 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-fe4781c3-a653-4916-816f-2add8619f5bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:08:43.594939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1116/2228] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:08:43 2022] DELETE /identity/v3/projects/0b5606e8f3884720be78850662d9647f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:08:43.600256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844dda9d-9395-47de-852e-90c687c3fbbf None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:43.600540 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844dda9d-9395-47de-852e-90c687c3fbbf None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:43.600814 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844dda9d-9395-47de-852e-90c687c3fbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:43.624421 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-844dda9d-9395-47de-852e-90c687c3fbbf None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:08:43.625598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-844dda9d-9395-47de-852e-90c687c3fbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:43.643082 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:08:43.698868 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1113/2229] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:08:43.707108 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be4dab-0930-47ba-b480-f86199152f09 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:43.707447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be4dab-0930-47ba-b480-f86199152f09 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:43.707755 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be4dab-0930-47ba-b480-f86199152f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:43.727743 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-10be4dab-0930-47ba-b480-f86199152f09 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:08:43.728928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-10be4dab-0930-47ba-b480-f86199152f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:43.737389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1117/2230] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:08:43.741600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844dda9d-9395-47de-852e-90c687c3fbbf None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:43.751475 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844dda9d-9395-47de-852e-90c687c3fbbf None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:43.756757 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2611b8-d226-4dc7-ad83-62d98b45314b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:43.757364 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2611b8-d226-4dc7-ad83-62d98b45314b admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:43.757693 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2611b8-d226-4dc7-ad83-62d98b45314b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:43.758066 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2611b8-d226-4dc7-ad83-62d98b45314b admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:43.758720 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2611b8-d226-4dc7-ad83-62d98b45314b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:08:43.759021 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2611b8-d226-4dc7-ad83-62d98b45314b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:43.760653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2611b8-d226-4dc7-ad83-62d98b45314b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:43.773434 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1114/2231] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:08:43.777619 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be4dab-0930-47ba-b480-f86199152f09 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:43.786975 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be4dab-0930-47ba-b480-f86199152f09 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:43.791972 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43dba2e-c3aa-4a21-8e0a-e65bd85a1795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:43.792329 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43dba2e-c3aa-4a21-8e0a-e65bd85a1795 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:43.792609 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43dba2e-c3aa-4a21-8e0a-e65bd85a1795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:43.792881 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43dba2e-c3aa-4a21-8e0a-e65bd85a1795 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:43.793519 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43dba2e-c3aa-4a21-8e0a-e65bd85a1795 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:43.795416 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43dba2e-c3aa-4a21-8e0a-e65bd85a1795 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:43.828929 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1118/2232] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:08:43 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:08:43.833536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2611b8-d226-4dc7-ad83-62d98b45314b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:43.843387 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2611b8-d226-4dc7-ad83-62d98b45314b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:43.900608 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb495bd-4941-412a-aa61-6f7645b5a45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:43.901250 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb495bd-4941-412a-aa61-6f7645b5a45a admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:43.901501 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb495bd-4941-412a-aa61-6f7645b5a45a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:43.901771 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb495bd-4941-412a-aa61-6f7645b5a45a admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:43.902388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb495bd-4941-412a-aa61-6f7645b5a45a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:43.904326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb495bd-4941-412a-aa61-6f7645b5a45a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:43.924943 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1115/2233] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:08:43 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:08:43.929210 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43dba2e-c3aa-4a21-8e0a-e65bd85a1795 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:43.939301 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43dba2e-c3aa-4a21-8e0a-e65bd85a1795 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:43.944457 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2271ba-9007-4255-9a9d-f95ec2991bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:43.945058 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2271ba-9007-4255-9a9d-f95ec2991bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:43.945340 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2271ba-9007-4255-9a9d-f95ec2991bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:43.945620 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2271ba-9007-4255-9a9d-f95ec2991bf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:43.946287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2271ba-9007-4255-9a9d-f95ec2991bf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:43.947929 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2271ba-9007-4255-9a9d-f95ec2991bf2 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:43.958600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1119/2234] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:08:43.962200 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb495bd-4941-412a-aa61-6f7645b5a45a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:43.972257 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb495bd-4941-412a-aa61-6f7645b5a45a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:43.977240 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b665c-5326-4c90-a487-e1e03e4517fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:43.977838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b665c-5326-4c90-a487-e1e03e4517fc admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:43.978120 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b665c-5326-4c90-a487-e1e03e4517fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:43.978430 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b665c-5326-4c90-a487-e1e03e4517fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:43.979118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b665c-5326-4c90-a487-e1e03e4517fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:43.980730 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b665c-5326-4c90-a487-e1e03e4517fc admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:43.991896 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1116/2235] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:08:43.996064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2271ba-9007-4255-9a9d-f95ec2991bf2 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:44.006754 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2271ba-9007-4255-9a9d-f95ec2991bf2 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:44.011531 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c93e75-6b87-4d7b-b007-48a62d10d107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:44.012734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c93e75-6b87-4d7b-b007-48a62d10d107 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:44.013193 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c93e75-6b87-4d7b-b007-48a62d10d107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:44.013605 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c93e75-6b87-4d7b-b007-48a62d10d107 admin admin] PATH_INFO: `/v3/projects/c3a4cb96eea945b8aeabe1ee77293241/users/933fbde6e9e0421eb7da2b7bd7850b84/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:44.029332 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c93e75-6b87-4d7b-b007-48a62d10d107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3a4cb96eea945b8aeabe1ee77293241, user_id=933fbde6e9e0421eb7da2b7bd7850b84, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:44.032944 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c93e75-6b87-4d7b-b007-48a62d10d107 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:44.047383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1120/2236] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:08:43 2022] PUT /identity/v3/projects/c3a4cb96eea945b8aeabe1ee77293241/users/933fbde6e9e0421eb7da2b7bd7850b84/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:08:44.116186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1558e97-73c8-47db-ac51-2345afba4788 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:44.116487 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1558e97-73c8-47db-ac51-2345afba4788 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:44.116762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1558e97-73c8-47db-ac51-2345afba4788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:44.145991 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a1558e97-73c8-47db-ac51-2345afba4788 None None] MFA Rules not processed for user `933fbde6e9e0421eb7da2b7bd7850b84`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:08:44.147222 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a1558e97-73c8-47db-ac51-2345afba4788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:44.211452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1117/2237] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:08:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:08:44.222239 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c93e75-6b87-4d7b-b007-48a62d10d107 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:44.223946 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-69c93e75-6b87-4d7b-b007-48a62d10d107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:44.245947 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:08:44.286001 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c93e75-6b87-4d7b-b007-48a62d10d107 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:08:44.289759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c47e36-2d9b-49ac-82bf-c5069acee895 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:08:44.290362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c47e36-2d9b-49ac-82bf-c5069acee895 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:44.290643 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c47e36-2d9b-49ac-82bf-c5069acee895 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:44.290917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c47e36-2d9b-49ac-82bf-c5069acee895 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:44.299892 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c47e36-2d9b-49ac-82bf-c5069acee895 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:08:44.301371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c47e36-2d9b-49ac-82bf-c5069acee895 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:08:44.310105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1121/2238] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:08:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:08:46.722706 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb2393-2ab7-42d7-9473-43cbd37629fd None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:08:46.723216 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb2393-2ab7-42d7-9473-43cbd37629fd None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:08:46.723311 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb2393-2ab7-42d7-9473-43cbd37629fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:08:46.742443 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-dbbb2393-2ab7-42d7-9473-43cbd37629fd None None] MFA Rules not processed for user `933fbde6e9e0421eb7da2b7bd7850b84`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:08:46.743649 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dbbb2393-2ab7-42d7-9473-43cbd37629fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:46.750617 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1118/2239] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:08:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:08:54.312110 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=17 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:08:56.736295 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:08:59.928789 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c47e36-2d9b-49ac-82bf-c5069acee895 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:08:59.930290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b6c47e36-2d9b-49ac-82bf-c5069acee895 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:08:59.996854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c47e36-2d9b-49ac-82bf-c5069acee895 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:09:00.001563 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494adad-814d-416f-9c24-02a834c91606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:09:00.002243 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494adad-814d-416f-9c24-02a834c91606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:00.003254 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494adad-814d-416f-9c24-02a834c91606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:00.003515 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494adad-814d-416f-9c24-02a834c91606 admin admin] PATH_INFO: `/v3/users/3dd372a44c6a47baa736d165a67dd495` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:00.004694 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494adad-814d-416f-9c24-02a834c91606 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dd372a44c6a47baa736d165a67dd495)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:09:00.006672 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494adad-814d-416f-9c24-02a834c91606 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:09:00.053068 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2494adad-814d-416f-9c24-02a834c91606 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dd372a44c6a47baa736d165a67dd495'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:00.053719 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2494adad-814d-416f-9c24-02a834c91606 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dd372a44c6a47baa736d165a67dd495'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:00.060913 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2494adad-814d-416f-9c24-02a834c91606 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dd372a44c6a47baa736d165a67dd495'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:00.068884 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2494adad-814d-416f-9c24-02a834c91606 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dd372a44c6a47baa736d165a67dd495'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:00.069519 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2494adad-814d-416f-9c24-02a834c91606 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dd372a44c6a47baa736d165a67dd495'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:00.086565 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1122/2240] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:08:59 2022] DELETE /identity/v3/users/3dd372a44c6a47baa736d165a67dd495 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:09:00.090970 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff632c43-2c21-4e5e-a9fb-c58f1b42338d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:00.091441 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff632c43-2c21-4e5e-a9fb-c58f1b42338d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:00.091543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff632c43-2c21-4e5e-a9fb-c58f1b42338d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:00.114751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ff632c43-2c21-4e5e-a9fb-c58f1b42338d None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:09:00.116243 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ff632c43-2c21-4e5e-a9fb-c58f1b42338d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:00.184069 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1119/2241] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:09:00.191498 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494adad-814d-416f-9c24-02a834c91606 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:09:00.192911 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2494adad-814d-416f-9c24-02a834c91606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:00.263909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494adad-814d-416f-9c24-02a834c91606 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:09:00.267486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4fd28-9e4e-4d41-87a6-ae6d99803cc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:09:00.268154 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4fd28-9e4e-4d41-87a6-ae6d99803cc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:00.268744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4fd28-9e4e-4d41-87a6-ae6d99803cc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:00.268744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4fd28-9e4e-4d41-87a6-ae6d99803cc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:00.278154 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab4fd28-9e4e-4d41-87a6-ae6d99803cc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:09:00.279641 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab4fd28-9e4e-4d41-87a6-ae6d99803cc4 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:09:00.289863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1123/2242] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:09:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:09:00.556095 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff632c43-2c21-4e5e-a9fb-c58f1b42338d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:09:00.557505 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ff632c43-2c21-4e5e-a9fb-c58f1b42338d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:00.569812 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff632c43-2c21-4e5e-a9fb-c58f1b42338d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:09:00.574387 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df562f88-7376-44d5-9427-673e618ddd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:09:00.575009 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df562f88-7376-44d5-9427-673e618ddd33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:00.575287 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df562f88-7376-44d5-9427-673e618ddd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:00.575588 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df562f88-7376-44d5-9427-673e618ddd33 admin admin] PATH_INFO: `/v3/projects/b70be8c825ad4f8e93295a5b249a289d` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:00.577916 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df562f88-7376-44d5-9427-673e618ddd33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b70be8c825ad4f8e93295a5b249a289d)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:09:00.579915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df562f88-7376-44d5-9427-673e618ddd33 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:09:00.629072 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-df562f88-7376-44d5-9427-673e618ddd33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b70be8c825ad4f8e93295a5b249a289d', 'request_id': 'req-df562f88-7376-44d5-9427-673e618ddd33'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:00.635792 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-df562f88-7376-44d5-9427-673e618ddd33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b70be8c825ad4f8e93295a5b249a289d', 'request_id': 'req-df562f88-7376-44d5-9427-673e618ddd33'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:00.641715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-df562f88-7376-44d5-9427-673e618ddd33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b70be8c825ad4f8e93295a5b249a289d', 'request_id': 'req-df562f88-7376-44d5-9427-673e618ddd33'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:00.652946 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-df562f88-7376-44d5-9427-673e618ddd33 admin admin] The token cache is being invalidate because project b70be8c825ad4f8e93295a5b249a289d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:09:00.653239 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-df562f88-7376-44d5-9427-673e618ddd33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:00.653877 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-df562f88-7376-44d5-9427-673e618ddd33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:00.655358 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1120/2243] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:09:00 2022] DELETE /identity/v3/projects/b70be8c825ad4f8e93295a5b249a289d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:09:00.661402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef103c-746e-47b7-a0a2-e3c2adc44fc7 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:00.661724 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef103c-746e-47b7-a0a2-e3c2adc44fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:00.662008 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef103c-746e-47b7-a0a2-e3c2adc44fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:00.681828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-3cef103c-746e-47b7-a0a2-e3c2adc44fc7 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:09:00.683044 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-3cef103c-746e-47b7-a0a2-e3c2adc44fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:00.700047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:09:00.745195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1124/2244] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:09:00.752934 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009d002-e1c7-4103-9a76-ba84d5ba470a None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:00.753220 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009d002-e1c7-4103-9a76-ba84d5ba470a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:00.753494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009d002-e1c7-4103-9a76-ba84d5ba470a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:00.773944 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-3009d002-e1c7-4103-9a76-ba84d5ba470a None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:09:00.775135 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3009d002-e1c7-4103-9a76-ba84d5ba470a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:00.782861 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1121/2245] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:09:00.787805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef103c-746e-47b7-a0a2-e3c2adc44fc7 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:09:00.797182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef103c-746e-47b7-a0a2-e3c2adc44fc7 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:09:00.802391 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c863b0-804a-442b-b801-7acfcaf53000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:09:00.803018 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c863b0-804a-442b-b801-7acfcaf53000 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:00.803298 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c863b0-804a-442b-b801-7acfcaf53000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:00.803586 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c863b0-804a-442b-b801-7acfcaf53000 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:00.804244 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c863b0-804a-442b-b801-7acfcaf53000 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:09:00.804545 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c863b0-804a-442b-b801-7acfcaf53000 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:09:00.806133 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c863b0-804a-442b-b801-7acfcaf53000 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:09:00.817816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1125/2246] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:09:00.821898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009d002-e1c7-4103-9a76-ba84d5ba470a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:09:00.831918 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009d002-e1c7-4103-9a76-ba84d5ba470a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:09:00.836695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6385588-cbd6-49b0-8356-bda31c27c290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:09:00.837316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6385588-cbd6-49b0-8356-bda31c27c290 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:00.837598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6385588-cbd6-49b0-8356-bda31c27c290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:00.837874 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6385588-cbd6-49b0-8356-bda31c27c290 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:00.838516 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6385588-cbd6-49b0-8356-bda31c27c290 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:09:00.840452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6385588-cbd6-49b0-8356-bda31c27c290 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:09:00.868395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1122/2247] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:09:00 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:09:00.872649 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c863b0-804a-442b-b801-7acfcaf53000 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:09:00.882739 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c863b0-804a-442b-b801-7acfcaf53000 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:09:00.943154 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b0e0a-03fa-45c1-b85f-14962ef4e5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:09:00.943682 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b0e0a-03fa-45c1-b85f-14962ef4e5ca admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:00.943682 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b0e0a-03fa-45c1-b85f-14962ef4e5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:00.943843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b0e0a-03fa-45c1-b85f-14962ef4e5ca admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:00.944472 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b0e0a-03fa-45c1-b85f-14962ef4e5ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:09:00.946390 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b0e0a-03fa-45c1-b85f-14962ef4e5ca admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:09:00.972293 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1126/2248] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:09:00 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:09:00.973736 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6385588-cbd6-49b0-8356-bda31c27c290 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:09:00.984267 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6385588-cbd6-49b0-8356-bda31c27c290 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:09:00.989174 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9aa9ff-ff2f-4ca6-a3a7-74b67741f059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:09:00.989784 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aa9ff-ff2f-4ca6-a3a7-74b67741f059 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:00.990068 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aa9ff-ff2f-4ca6-a3a7-74b67741f059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:00.990349 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aa9ff-ff2f-4ca6-a3a7-74b67741f059 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:00.990993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9aa9ff-ff2f-4ca6-a3a7-74b67741f059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:09:00.992656 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9aa9ff-ff2f-4ca6-a3a7-74b67741f059 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:09:01.004157 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1123/2249] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:09:01.008893 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b0e0a-03fa-45c1-b85f-14962ef4e5ca admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:09:01.020532 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b0e0a-03fa-45c1-b85f-14962ef4e5ca admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:09:01.026082 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1291c-0063-430f-a3d8-69b8d2f8d706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:09:01.026918 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1291c-0063-430f-a3d8-69b8d2f8d706 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:01.027280 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1291c-0063-430f-a3d8-69b8d2f8d706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:01.027673 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1291c-0063-430f-a3d8-69b8d2f8d706 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:01.028488 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1291c-0063-430f-a3d8-69b8d2f8d706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:09:01.030628 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1291c-0063-430f-a3d8-69b8d2f8d706 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:09:01.046487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1127/2250] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:09:01.051033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9aa9ff-ff2f-4ca6-a3a7-74b67741f059 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:09:01.061689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9aa9ff-ff2f-4ca6-a3a7-74b67741f059 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:09:01.067044 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a7803-1151-4699-b4d5-4911c8ac1d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:09:01.067829 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a7803-1151-4699-b4d5-4911c8ac1d2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:01.068177 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a7803-1151-4699-b4d5-4911c8ac1d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:01.068525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a7803-1151-4699-b4d5-4911c8ac1d2f admin admin] PATH_INFO: `/v3/projects/9dafb4e09aea4066994536051a968935/users/f42079fa43da420e8483c31fd5ed70dc/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:01.087338 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61a7803-1151-4699-b4d5-4911c8ac1d2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dafb4e09aea4066994536051a968935, user_id=f42079fa43da420e8483c31fd5ed70dc, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:09:01.090582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61a7803-1151-4699-b4d5-4911c8ac1d2f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:09:01.107594 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1124/2251] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:09:01 2022] PUT /identity/v3/projects/9dafb4e09aea4066994536051a968935/users/f42079fa43da420e8483c31fd5ed70dc/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:09:01.119109 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28bf21-ccee-4fba-9658-a1d7c3b7880d None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:01.119435 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28bf21-ccee-4fba-9658-a1d7c3b7880d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:01.119756 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28bf21-ccee-4fba-9658-a1d7c3b7880d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:01.152828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-5e28bf21-ccee-4fba-9658-a1d7c3b7880d None None] MFA Rules not processed for user `f42079fa43da420e8483c31fd5ed70dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:09:01.154172 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5e28bf21-ccee-4fba-9658-a1d7c3b7880d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:01.224917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1128/2252] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:09:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:09:01.236291 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a7803-1151-4699-b4d5-4911c8ac1d2f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:09:01.238151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d61a7803-1151-4699-b4d5-4911c8ac1d2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:01.261136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:09:01.298587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a7803-1151-4699-b4d5-4911c8ac1d2f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:09:01.302505 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af3d36-d935-4a76-b7ea-2a71954c71e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:09:01.303112 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af3d36-d935-4a76-b7ea-2a71954c71e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:01.303417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af3d36-d935-4a76-b7ea-2a71954c71e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:01.303724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af3d36-d935-4a76-b7ea-2a71954c71e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:01.311926 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af3d36-d935-4a76-b7ea-2a71954c71e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:09:01.313291 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af3d36-d935-4a76-b7ea-2a71954c71e8 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:09:01.322735 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1125/2253] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:09:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:09:06.292027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792366e-ae04-479c-be39-d68ea659810d None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:06.292546 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792366e-ae04-479c-be39-d68ea659810d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:06.292546 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792366e-ae04-479c-be39-d68ea659810d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:06.312808 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-4792366e-ae04-479c-be39-d68ea659810d None None] MFA Rules not processed for user `0b9c17d96f3d44589cee145c8ec2e97d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:09:06.314015 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4792366e-ae04-479c-be39-d68ea659810d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:06.325994 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:09:06.374425 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1129/2254] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:09:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:09:11.319280 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=17 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:09:16.367670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=17 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:09:32.656031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98884fdb-3c67-44cb-be9a-6c707921cc8f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:32.656606 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98884fdb-3c67-44cb-be9a-6c707921cc8f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:32.656606 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98884fdb-3c67-44cb-be9a-6c707921cc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:32.678571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-98884fdb-3c67-44cb-be9a-6c707921cc8f None None] MFA Rules not processed for user `51768890f19140b6819ef89f5bf610c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:09:32.679819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-98884fdb-3c67-44cb-be9a-6c707921cc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:32.755130 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1126/2255] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:09:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:09:32.842840 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792366e-ae04-479c-be39-d68ea659810d None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:09:32.912948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792366e-ae04-479c-be39-d68ea659810d None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:09:32.917457 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db28fe4-5459-4787-8fb9-8d6cdc70591a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:09:32.918090 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db28fe4-5459-4787-8fb9-8d6cdc70591a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:32.918399 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db28fe4-5459-4787-8fb9-8d6cdc70591a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:32.918676 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db28fe4-5459-4787-8fb9-8d6cdc70591a admin admin] PATH_INFO: `/v3/users/f42079fa43da420e8483c31fd5ed70dc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:32.919847 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db28fe4-5459-4787-8fb9-8d6cdc70591a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f42079fa43da420e8483c31fd5ed70dc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:09:32.921798 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db28fe4-5459-4787-8fb9-8d6cdc70591a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:09:32.970724 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6db28fe4-5459-4787-8fb9-8d6cdc70591a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f42079fa43da420e8483c31fd5ed70dc'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:32.971388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6db28fe4-5459-4787-8fb9-8d6cdc70591a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f42079fa43da420e8483c31fd5ed70dc'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:32.982431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6db28fe4-5459-4787-8fb9-8d6cdc70591a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f42079fa43da420e8483c31fd5ed70dc'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:32.989859 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6db28fe4-5459-4787-8fb9-8d6cdc70591a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f42079fa43da420e8483c31fd5ed70dc'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:32.990758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6db28fe4-5459-4787-8fb9-8d6cdc70591a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f42079fa43da420e8483c31fd5ed70dc'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:33.008972 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1130/2256] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:09:32 2022] DELETE /identity/v3/users/f42079fa43da420e8483c31fd5ed70dc => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:09:33.013795 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86683cbe-cece-4a4d-b62c-e80e275d4d26 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:33.014148 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86683cbe-cece-4a4d-b62c-e80e275d4d26 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:33.014485 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86683cbe-cece-4a4d-b62c-e80e275d4d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:33.035430 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-86683cbe-cece-4a4d-b62c-e80e275d4d26 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:09:33.036762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-86683cbe-cece-4a4d-b62c-e80e275d4d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:33.098701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1127/2257] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:09:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:09:33.108789 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db28fe4-5459-4787-8fb9-8d6cdc70591a admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:09:33.110255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6db28fe4-5459-4787-8fb9-8d6cdc70591a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:33.182691 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db28fe4-5459-4787-8fb9-8d6cdc70591a admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:09:33.186562 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ec392-faab-4325-bcd4-4774b9468f18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:09:33.187179 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ec392-faab-4325-bcd4-4774b9468f18 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:33.187464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ec392-faab-4325-bcd4-4774b9468f18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:33.187765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ec392-faab-4325-bcd4-4774b9468f18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:33.197874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ec392-faab-4325-bcd4-4774b9468f18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:09:33.199338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ec392-faab-4325-bcd4-4774b9468f18 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:09:33.210654 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1131/2258] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:09:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:09:33.536106 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86683cbe-cece-4a4d-b62c-e80e275d4d26 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:09:33.537528 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-86683cbe-cece-4a4d-b62c-e80e275d4d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:33.555670 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86683cbe-cece-4a4d-b62c-e80e275d4d26 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:09:33.571700 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0e36c-27b7-4582-b6fd-4536fb07e902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:09:33.573408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0e36c-27b7-4582-b6fd-4536fb07e902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:33.574622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0e36c-27b7-4582-b6fd-4536fb07e902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:33.576149 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0e36c-27b7-4582-b6fd-4536fb07e902 admin admin] PATH_INFO: `/v3/projects/9dafb4e09aea4066994536051a968935` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:33.578992 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0e36c-27b7-4582-b6fd-4536fb07e902 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dafb4e09aea4066994536051a968935)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:09:33.581115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0e36c-27b7-4582-b6fd-4536fb07e902 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:09:33.678262 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-faa0e36c-27b7-4582-b6fd-4536fb07e902 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dafb4e09aea4066994536051a968935', 'request_id': 'req-faa0e36c-27b7-4582-b6fd-4536fb07e902'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:33.690962 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-faa0e36c-27b7-4582-b6fd-4536fb07e902 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dafb4e09aea4066994536051a968935', 'request_id': 'req-faa0e36c-27b7-4582-b6fd-4536fb07e902'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:33.703653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-faa0e36c-27b7-4582-b6fd-4536fb07e902 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dafb4e09aea4066994536051a968935', 'request_id': 'req-faa0e36c-27b7-4582-b6fd-4536fb07e902'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:33.734951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-faa0e36c-27b7-4582-b6fd-4536fb07e902 admin admin] The token cache is being invalidate because project 9dafb4e09aea4066994536051a968935 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:09:33.735383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-faa0e36c-27b7-4582-b6fd-4536fb07e902 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:33.736609 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-faa0e36c-27b7-4582-b6fd-4536fb07e902 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:33.740827 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1128/2259] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:09:33 2022] DELETE /identity/v3/projects/9dafb4e09aea4066994536051a968935 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:09:33.760404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1c001-5609-48db-b7d6-73ed2212b016 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:33.761116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1c001-5609-48db-b7d6-73ed2212b016 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:33.761682 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1c001-5609-48db-b7d6-73ed2212b016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:33.789277 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-44d1c001-5609-48db-b7d6-73ed2212b016 None None] MFA Rules not processed for user `ceccc909529042539dd2d5015058d8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:09:33.791625 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-44d1c001-5609-48db-b7d6-73ed2212b016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:33.816875 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:09:33.879250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1132/2260] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 01:09:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:09:33.938004 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0e36c-27b7-4582-b6fd-4536fb07e902 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:09:33.939922 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-faa0e36c-27b7-4582-b6fd-4536fb07e902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:33.983076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:09:34.035355 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0e36c-27b7-4582-b6fd-4536fb07e902 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:09:34.040474 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efd931-03b9-4e6c-859e-444e2184e8ba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e52776d9b4894af9b85a24136a8255e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:09:34.041147 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efd931-03b9-4e6c-859e-444e2184e8ba service swift] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:34.041491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efd931-03b9-4e6c-859e-444e2184e8ba service swift] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:34.041766 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efd931-03b9-4e6c-859e-444e2184e8ba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:34.056766 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efd931-03b9-4e6c-859e-444e2184e8ba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:09:34.058356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efd931-03b9-4e6c-859e-444e2184e8ba service swift] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:09:34.067015 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1129/2261] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 01:09:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 01:09:34.666770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e7846-11e1-4cae-a760-2fc28897273f None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:34.671666 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e7846-11e1-4cae-a760-2fc28897273f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:34.672270 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e7846-11e1-4cae-a760-2fc28897273f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:34.694014 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-4f4e7846-11e1-4cae-a760-2fc28897273f None None] MFA Rules not processed for user `ceccc909529042539dd2d5015058d8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:09:34.695375 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4f4e7846-11e1-4cae-a760-2fc28897273f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:34.702931 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1133/2262] 10.222.0.42 () {58 vars in 967 bytes} [Wed Sep 14 01:09:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:09:34.717763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efd931-03b9-4e6c-859e-444e2184e8ba service swift] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:09:34.730364 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efd931-03b9-4e6c-859e-444e2184e8ba service swift] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:09:34.742269 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b727933-4444-4a42-a5d6-757f8a960ad8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e52776d9b4894af9b85a24136a8255e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:09:34.743096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b727933-4444-4a42-a5d6-757f8a960ad8 service swift] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:34.743457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b727933-4444-4a42-a5d6-757f8a960ad8 service swift] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:34.743844 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b727933-4444-4a42-a5d6-757f8a960ad8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:34.755426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b727933-4444-4a42-a5d6-757f8a960ad8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:09:34.757171 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b727933-4444-4a42-a5d6-757f8a960ad8 service swift] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:09:34.767817 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1130/2263] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 01:09:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 01:09:35.045702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7a330-b873-4a8d-a46b-d1a8dd7e20b1 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:35.047422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7a330-b873-4a8d-a46b-d1a8dd7e20b1 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:35.048018 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7a330-b873-4a8d-a46b-d1a8dd7e20b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:35.071393 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-cbd7a330-b873-4a8d-a46b-d1a8dd7e20b1 None None] MFA Rules not processed for user `ceccc909529042539dd2d5015058d8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:09:35.072668 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-cbd7a330-b873-4a8d-a46b-d1a8dd7e20b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:35.086434 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1134/2264] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 01:09:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:09:35.100814 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b727933-4444-4a42-a5d6-757f8a960ad8 service swift] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:09:35.117446 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b727933-4444-4a42-a5d6-757f8a960ad8 service swift] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:09:35.122786 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83380a6-17c4-4e3a-8e01-0df6657af1c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e52776d9b4894af9b85a24136a8255e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:09:35.123386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83380a6-17c4-4e3a-8e01-0df6657af1c9 service swift] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:35.123702 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83380a6-17c4-4e3a-8e01-0df6657af1c9 service swift] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:35.124059 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83380a6-17c4-4e3a-8e01-0df6657af1c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:35.138607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83380a6-17c4-4e3a-8e01-0df6657af1c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:09:35.139846 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83380a6-17c4-4e3a-8e01-0df6657af1c9 service swift] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:09:35.150388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1131/2265] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 01:09:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 01:09:35.659545 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce49e2-479e-4a06-b831-85f5ee9a32f5 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:35.660606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce49e2-479e-4a06-b831-85f5ee9a32f5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:35.660888 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce49e2-479e-4a06-b831-85f5ee9a32f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:35.689913 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d8ce49e2-479e-4a06-b831-85f5ee9a32f5 None None] MFA Rules not processed for user `ceccc909529042539dd2d5015058d8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:09:35.692625 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d8ce49e2-479e-4a06-b831-85f5ee9a32f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:35.715685 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1135/2266] 10.222.0.42 () {58 vars in 967 bytes} [Wed Sep 14 01:09:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:09:35.742521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83380a6-17c4-4e3a-8e01-0df6657af1c9 service swift] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:09:35.774993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83380a6-17c4-4e3a-8e01-0df6657af1c9 service swift] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:09:35.781173 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be72130-d541-4a07-b585-353532b054a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e52776d9b4894af9b85a24136a8255e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:09:35.782723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be72130-d541-4a07-b585-353532b054a0 service swift] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:35.783093 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be72130-d541-4a07-b585-353532b054a0 service swift] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:35.783656 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be72130-d541-4a07-b585-353532b054a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:35.806507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be72130-d541-4a07-b585-353532b054a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:09:35.806507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be72130-d541-4a07-b585-353532b054a0 service swift] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:09:35.835296 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1132/2267] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 01:09:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 01:09:38.068846 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce49e2-479e-4a06-b831-85f5ee9a32f5 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:09:38.070367 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d8ce49e2-479e-4a06-b831-85f5ee9a32f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:38.137148 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce49e2-479e-4a06-b831-85f5ee9a32f5 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:09:38.141718 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448f2c0-9324-4001-aa5d-9298a6422446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:09:38.142353 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448f2c0-9324-4001-aa5d-9298a6422446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:38.142638 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448f2c0-9324-4001-aa5d-9298a6422446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:38.142914 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448f2c0-9324-4001-aa5d-9298a6422446 admin admin] PATH_INFO: `/v3/users/933fbde6e9e0421eb7da2b7bd7850b84` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:38.144079 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448f2c0-9324-4001-aa5d-9298a6422446 admin admin] RBAC: Authorizing `identity:delete_user(user_id=933fbde6e9e0421eb7da2b7bd7850b84)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:09:38.146673 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448f2c0-9324-4001-aa5d-9298a6422446 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:09:38.196253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-0448f2c0-9324-4001-aa5d-9298a6422446 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '933fbde6e9e0421eb7da2b7bd7850b84'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:38.196947 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-0448f2c0-9324-4001-aa5d-9298a6422446 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '933fbde6e9e0421eb7da2b7bd7850b84'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:38.205090 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-0448f2c0-9324-4001-aa5d-9298a6422446 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '933fbde6e9e0421eb7da2b7bd7850b84'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:38.213524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-0448f2c0-9324-4001-aa5d-9298a6422446 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '933fbde6e9e0421eb7da2b7bd7850b84'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:38.214217 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-0448f2c0-9324-4001-aa5d-9298a6422446 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '933fbde6e9e0421eb7da2b7bd7850b84'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:38.233027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1136/2268] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:09:38 2022] DELETE /identity/v3/users/933fbde6e9e0421eb7da2b7bd7850b84 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:09:38.237274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e816a-f192-492f-bca1-a3647921f3b3 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:38.237760 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e816a-f192-492f-bca1-a3647921f3b3 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:38.237852 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e816a-f192-492f-bca1-a3647921f3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:38.255895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448f2c0-9324-4001-aa5d-9298a6422446 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:09:38.257330 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-6a5e816a-f192-492f-bca1-a3647921f3b3 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:09:38.257512 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0448f2c0-9324-4001-aa5d-9298a6422446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:38.258674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6a5e816a-f192-492f-bca1-a3647921f3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:38.320250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:09:38.351736 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1133/2269] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:09:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:09:38.356081 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448f2c0-9324-4001-aa5d-9298a6422446 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:09:38.358654 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e816a-f192-492f-bca1-a3647921f3b3 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:09:38.363521 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47659a8-9403-4aaf-947d-70635617d9ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:09:38.364535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47659a8-9403-4aaf-947d-70635617d9ad service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:38.365112 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47659a8-9403-4aaf-947d-70635617d9ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:38.365508 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47659a8-9403-4aaf-947d-70635617d9ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:38.367952 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f47659a8-9403-4aaf-947d-70635617d9ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:38.378891 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e816a-f192-492f-bca1-a3647921f3b3 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:09:38.383391 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ac02e-6834-4c95-bd00-a7efa8ed58b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:09:38.383842 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ac02e-6834-4c95-bd00-a7efa8ed58b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:38.384261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ac02e-6834-4c95-bd00-a7efa8ed58b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:38.384550 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ac02e-6834-4c95-bd00-a7efa8ed58b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:38.399217 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ac02e-6834-4c95-bd00-a7efa8ed58b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:09:38.400718 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ac02e-6834-4c95-bd00-a7efa8ed58b2 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:09:38.413874 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1134/2270] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:09:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:09:38.430237 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ac02e-6834-4c95-bd00-a7efa8ed58b2 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:09:38.434414 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0f1ac02e-6834-4c95-bd00-a7efa8ed58b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:38.460358 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ac02e-6834-4c95-bd00-a7efa8ed58b2 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:09:38.467357 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558288c-3211-441b-8d7f-00641a0d5cf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:09:38.468227 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558288c-3211-441b-8d7f-00641a0d5cf7 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:38.468656 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558288c-3211-441b-8d7f-00641a0d5cf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:38.469281 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558288c-3211-441b-8d7f-00641a0d5cf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:38.475709 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e558288c-3211-441b-8d7f-00641a0d5cf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:38.499835 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47659a8-9403-4aaf-947d-70635617d9ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:09:38.501746 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47659a8-9403-4aaf-947d-70635617d9ad service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:09:38.514944 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1137/2271] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:09:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:09:38.553617 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:09:38.570469 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558288c-3211-441b-8d7f-00641a0d5cf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:09:38.572105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558288c-3211-441b-8d7f-00641a0d5cf7 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:09:38.583833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1135/2272] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:09:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:09:38.819992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47659a8-9403-4aaf-947d-70635617d9ad service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:09:38.821492 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f47659a8-9403-4aaf-947d-70635617d9ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:38.837113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47659a8-9403-4aaf-947d-70635617d9ad service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:09:38.841968 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab16e88-28b1-4dc8-a2cc-4db28eabb508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:09:38.842608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab16e88-28b1-4dc8-a2cc-4db28eabb508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:38.842892 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab16e88-28b1-4dc8-a2cc-4db28eabb508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:38.843510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab16e88-28b1-4dc8-a2cc-4db28eabb508 admin admin] PATH_INFO: `/v3/projects/c3a4cb96eea945b8aeabe1ee77293241` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:38.844339 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab16e88-28b1-4dc8-a2cc-4db28eabb508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3a4cb96eea945b8aeabe1ee77293241)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:09:38.846320 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab16e88-28b1-4dc8-a2cc-4db28eabb508 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:09:38.901141 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4ab16e88-28b1-4dc8-a2cc-4db28eabb508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3a4cb96eea945b8aeabe1ee77293241', 'request_id': 'req-4ab16e88-28b1-4dc8-a2cc-4db28eabb508'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:38.909875 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4ab16e88-28b1-4dc8-a2cc-4db28eabb508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3a4cb96eea945b8aeabe1ee77293241', 'request_id': 'req-4ab16e88-28b1-4dc8-a2cc-4db28eabb508'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:38.916911 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4ab16e88-28b1-4dc8-a2cc-4db28eabb508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3a4cb96eea945b8aeabe1ee77293241', 'request_id': 'req-4ab16e88-28b1-4dc8-a2cc-4db28eabb508'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:38.929010 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4ab16e88-28b1-4dc8-a2cc-4db28eabb508 admin admin] The token cache is being invalidate because project c3a4cb96eea945b8aeabe1ee77293241 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:09:38.929327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4ab16e88-28b1-4dc8-a2cc-4db28eabb508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:38.930028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4ab16e88-28b1-4dc8-a2cc-4db28eabb508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:09:38.931522 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1138/2273] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:09:38 2022] DELETE /identity/v3/projects/c3a4cb96eea945b8aeabe1ee77293241 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:09:48.579324 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:09:48.925470 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:09:51.301840 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558288c-3211-441b-8d7f-00641a0d5cf7 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:09:51.311797 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558288c-3211-441b-8d7f-00641a0d5cf7 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:09:51.364373 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b5bf4-039a-46f9-9fb2-cd1eff5e125c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:09:51.365009 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b5bf4-039a-46f9-9fb2-cd1eff5e125c service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:51.365292 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b5bf4-039a-46f9-9fb2-cd1eff5e125c service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:51.365600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b5bf4-039a-46f9-9fb2-cd1eff5e125c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:51.367498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-366b5bf4-039a-46f9-9fb2-cd1eff5e125c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:51.426364 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b5bf4-039a-46f9-9fb2-cd1eff5e125c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:09:51.427932 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b5bf4-039a-46f9-9fb2-cd1eff5e125c service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:09:51.436892 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1136/2274] 10.222.0.42 () {60 vars in 1396 bytes} [Wed Sep 14 01:09:51 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3254 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:09:55.183936 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab16e88-28b1-4dc8-a2cc-4db28eabb508 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:09:55.185267 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4ab16e88-28b1-4dc8-a2cc-4db28eabb508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:55.272137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab16e88-28b1-4dc8-a2cc-4db28eabb508 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:09:55.277726 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8b9677-57bc-4407-a73e-b5713bb81d1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:09:55.278344 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b9677-57bc-4407-a73e-b5713bb81d1e service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:55.278626 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b9677-57bc-4407-a73e-b5713bb81d1e service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:55.278904 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b9677-57bc-4407-a73e-b5713bb81d1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:55.280688 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7b8b9677-57bc-4407-a73e-b5713bb81d1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:55.345317 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8b9677-57bc-4407-a73e-b5713bb81d1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:09:55.347113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8b9677-57bc-4407-a73e-b5713bb81d1e service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:09:55.356630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1139/2275] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:09:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:09:59.935959 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b5bf4-039a-46f9-9fb2-cd1eff5e125c service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:09:59.937476 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-366b5bf4-039a-46f9-9fb2-cd1eff5e125c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:09:59.953336 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b5bf4-039a-46f9-9fb2-cd1eff5e125c service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:09:59.959379 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb14b429-b168-4165-b699-107043c7d642 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:09:59.960008 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14b429-b168-4165-b699-107043c7d642 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:09:59.960287 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14b429-b168-4165-b699-107043c7d642 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:09:59.960559 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14b429-b168-4165-b699-107043c7d642 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:09:59.963291 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fb14b429-b168-4165-b699-107043c7d642 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:10:00.029483 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb14b429-b168-4165-b699-107043c7d642 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:10:00.031390 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb14b429-b168-4165-b699-107043c7d642 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:10:00.041810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1137/2276] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:09:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:10:02.783185 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f545f-56cf-45d4-847f-1fc4ef74e60a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:02.783684 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f545f-56cf-45d4-847f-1fc4ef74e60a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:02.783778 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f545f-56cf-45d4-847f-1fc4ef74e60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:02.803427 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-7a6f545f-56cf-45d4-847f-1fc4ef74e60a None None] MFA Rules not processed for user `ceccc909529042539dd2d5015058d8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:10:02.804714 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7a6f545f-56cf-45d4-847f-1fc4ef74e60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:10:02.860528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1140/2277] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 01:10:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:10:02.873837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb14b429-b168-4165-b699-107043c7d642 service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:10:02.875268 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fb14b429-b168-4165-b699-107043c7d642 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:10:02.933275 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:10:02.950442 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb14b429-b168-4165-b699-107043c7d642 service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:10:02.955925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc6df0-e584-4802-a189-fef7afe7f779 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e52776d9b4894af9b85a24136a8255e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:10:02.956480 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc6df0-e584-4802-a189-fef7afe7f779 service swift] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:02.956881 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc6df0-e584-4802-a189-fef7afe7f779 service swift] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:02.957307 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc6df0-e584-4802-a189-fef7afe7f779 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:02.965647 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdc6df0-e584-4802-a189-fef7afe7f779 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:10:02.967251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdc6df0-e584-4802-a189-fef7afe7f779 service swift] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:10:02.976189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1138/2278] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 01:10:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 01:10:12.853753 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=81 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:10:12.972900 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=8 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:10:19.080494 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f545f-56cf-45d4-847f-1fc4ef74e60a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:10:19.082491 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7a6f545f-56cf-45d4-847f-1fc4ef74e60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:10:19.093503 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f545f-56cf-45d4-847f-1fc4ef74e60a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:10:19.097939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72e359-f186-409e-9d8c-e783b1367a72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:10:19.098537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72e359-f186-409e-9d8c-e783b1367a72 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:19.098818 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72e359-f186-409e-9d8c-e783b1367a72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:19.099090 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72e359-f186-409e-9d8c-e783b1367a72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:19.100772 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2a72e359-f186-409e-9d8c-e783b1367a72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:10:19.109980 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72e359-f186-409e-9d8c-e783b1367a72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:10:19.111394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72e359-f186-409e-9d8c-e783b1367a72 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:10:19.121266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1141/2279] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:10:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:10:22.652222 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc6df0-e584-4802-a189-fef7afe7f779 service swift] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:10:22.653749 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2fdc6df0-e584-4802-a189-fef7afe7f779 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:10:22.665603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc6df0-e584-4802-a189-fef7afe7f779 service swift] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:10:22.669144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf1792-cce9-4f79-8eac-c563744cee21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:10:22.669742 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf1792-cce9-4f79-8eac-c563744cee21 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:22.670021 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf1792-cce9-4f79-8eac-c563744cee21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:22.670293 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf1792-cce9-4f79-8eac-c563744cee21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:22.672025 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bcbf1792-cce9-4f79-8eac-c563744cee21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:10:22.682954 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf1792-cce9-4f79-8eac-c563744cee21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:10:22.684450 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf1792-cce9-4f79-8eac-c563744cee21 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:10:22.693483 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1139/2280] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:10:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:10:22.815732 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ead72e-5e11-497d-afc4-4448927e13cc None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:22.816863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ead72e-5e11-497d-afc4-4448927e13cc None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:22.816863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ead72e-5e11-497d-afc4-4448927e13cc None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:22.818216 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1142/2281] 10.222.0.42 () {54 vars in 921 bytes} [Wed Sep 14 01:10:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:10:22.824871 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab2a83-1de7-4a64-b7ae-bd0165ac3386 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:22.825188 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab2a83-1de7-4a64-b7ae-bd0165ac3386 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:22.825454 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab2a83-1de7-4a64-b7ae-bd0165ac3386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:22.846602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-76ab2a83-1de7-4a64-b7ae-bd0165ac3386 None None] MFA Rules not processed for user `5e0fe2e84f6748c992c9beedf2f5d49b`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:10:22.849061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-76ab2a83-1de7-4a64-b7ae-bd0165ac3386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:10:22.858725 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1140/2282] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Sep 14 01:10:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:10:22.870095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ead72e-5e11-497d-afc4-4448927e13cc None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:10:22.871951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-05ead72e-5e11-497d-afc4-4448927e13cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:10:22.934446 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ead72e-5e11-497d-afc4-4448927e13cc None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:10:22.938888 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d1191-d249-4d4a-b2f3-84ba666c8140 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89ff357209194276bc079d0c05898747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:10:22.939507 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d1191-d249-4d4a-b2f3-84ba666c8140 service glance] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:22.939825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d1191-d249-4d4a-b2f3-84ba666c8140 service glance] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:22.940106 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d1191-d249-4d4a-b2f3-84ba666c8140 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:22.951321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d1191-d249-4d4a-b2f3-84ba666c8140 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:10:22.952725 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d1191-d249-4d4a-b2f3-84ba666c8140 service glance] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:10:22.963858 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1143/2283] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:10:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:10:23.036222 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dcf70-8265-4069-868a-1ded06687eca None None] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:23.036614 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dcf70-8265-4069-868a-1ded06687eca None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:23.036957 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dcf70-8265-4069-868a-1ded06687eca None None] PATH_INFO: `/` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:23.038411 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1141/2284] 10.222.0.42 () {54 vars in 921 bytes} [Wed Sep 14 01:10:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:10:23.043192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e4ad9-c457-44ff-a85f-7b53bb29f868 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:23.043506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e4ad9-c457-44ff-a85f-7b53bb29f868 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:23.043800 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e4ad9-c457-44ff-a85f-7b53bb29f868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:23.066935 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-995e4ad9-c457-44ff-a85f-7b53bb29f868 None None] MFA Rules not processed for user `4d64f9bca7c546e4ba7f54c1c47d665e`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:10:23.068185 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-995e4ad9-c457-44ff-a85f-7b53bb29f868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:10:23.075451 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1144/2285] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Sep 14 01:10:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:10:23.087262 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620dcf70-8265-4069-868a-1ded06687eca None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:10:23.100960 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620dcf70-8265-4069-868a-1ded06687eca None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:10:23.104174 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df305270-a1b8-4124-9f77-ecf4e2f8b6df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:10:23.104781 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df305270-a1b8-4124-9f77-ecf4e2f8b6df service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:23.105065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df305270-a1b8-4124-9f77-ecf4e2f8b6df service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:23.105343 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df305270-a1b8-4124-9f77-ecf4e2f8b6df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:23.112661 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df305270-a1b8-4124-9f77-ecf4e2f8b6df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:10:23.114955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df305270-a1b8-4124-9f77-ecf4e2f8b6df service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:10:23.123018 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1142/2286] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:10:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:10:23.219771 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba95afd-4de6-44ca-96ec-553864321c61 None None] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:23.220065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba95afd-4de6-44ca-96ec-553864321c61 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:23.220344 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba95afd-4de6-44ca-96ec-553864321c61 None None] PATH_INFO: `/` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:23.221731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1145/2287] 10.222.0.42 () {54 vars in 921 bytes} [Wed Sep 14 01:10:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:10:23.227902 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27cab1-6e29-4acb-a237-cc07643b2980 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:23.228190 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27cab1-6e29-4acb-a237-cc07643b2980 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:23.228466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27cab1-6e29-4acb-a237-cc07643b2980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:23.248605 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0c27cab1-6e29-4acb-a237-cc07643b2980 None None] MFA Rules not processed for user `5e0fe2e84f6748c992c9beedf2f5d49b`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:10:23.249786 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0c27cab1-6e29-4acb-a237-cc07643b2980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:10:23.259184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1143/2288] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Sep 14 01:10:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:10:23.269967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba95afd-4de6-44ca-96ec-553864321c61 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:10:23.280954 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba95afd-4de6-44ca-96ec-553864321c61 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:10:23.284523 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ede78d-790f-450f-a425-d7b708bde8b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:10:23.285738 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ede78d-790f-450f-a425-d7b708bde8b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:23.286022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ede78d-790f-450f-a425-d7b708bde8b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:23.286294 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ede78d-790f-450f-a425-d7b708bde8b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:23.296253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ede78d-790f-450f-a425-d7b708bde8b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:10:23.297689 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ede78d-790f-450f-a425-d7b708bde8b6 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:10:23.309047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1146/2289] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:10:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:10:23.370343 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27cab1-6e29-4acb-a237-cc07643b2980 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:10:23.371765 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0c27cab1-6e29-4acb-a237-cc07643b2980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:10:23.439465 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27cab1-6e29-4acb-a237-cc07643b2980 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:10:23.445107 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597db4e7-b1d0-4c72-9606-da1d8380cabd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18798838de284c84b5b838b3530a2c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:10:23.445710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597db4e7-b1d0-4c72-9606-da1d8380cabd service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:23.446010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597db4e7-b1d0-4c72-9606-da1d8380cabd service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:23.446264 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597db4e7-b1d0-4c72-9606-da1d8380cabd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:23.448037 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-597db4e7-b1d0-4c72-9606-da1d8380cabd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:10:23.459159 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597db4e7-b1d0-4c72-9606-da1d8380cabd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:10:23.460829 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597db4e7-b1d0-4c72-9606-da1d8380cabd service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:10:23.470436 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1144/2290] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:10:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:10:30.844549 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e656a3-9013-4318-89d8-62557227fb8d None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:30.845071 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e656a3-9013-4318-89d8-62557227fb8d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:30.845241 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e656a3-9013-4318-89d8-62557227fb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:30.869096 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-29e656a3-9013-4318-89d8-62557227fb8d None None] MFA Rules not processed for user `ceccc909529042539dd2d5015058d8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:10:30.870332 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-29e656a3-9013-4318-89d8-62557227fb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:10:30.877596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1147/2291] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 01:10:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:10:30.890302 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597db4e7-b1d0-4c72-9606-da1d8380cabd service placement] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:10:30.902249 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597db4e7-b1d0-4c72-9606-da1d8380cabd service placement] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:10:30.907956 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cbe30-1b5a-41f2-a6b5-21b4e4ee085d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e52776d9b4894af9b85a24136a8255e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:10:30.908552 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cbe30-1b5a-41f2-a6b5-21b4e4ee085d service swift] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:30.908832 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cbe30-1b5a-41f2-a6b5-21b4e4ee085d service swift] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:30.909677 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cbe30-1b5a-41f2-a6b5-21b4e4ee085d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:30.918562 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34cbe30-1b5a-41f2-a6b5-21b4e4ee085d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:10:30.920275 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34cbe30-1b5a-41f2-a6b5-21b4e4ee085d service swift] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:10:30.930157 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1145/2292] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 01:10:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 14 01:10:39.545953 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e656a3-9013-4318-89d8-62557227fb8d None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:10:39.548262 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-29e656a3-9013-4318-89d8-62557227fb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:10:39.643919 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e656a3-9013-4318-89d8-62557227fb8d None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:10:39.650137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d424fe-15df-4ec7-9647-a556ed286159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:10:39.650805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d424fe-15df-4ec7-9647-a556ed286159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:39.651092 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d424fe-15df-4ec7-9647-a556ed286159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:39.651398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d424fe-15df-4ec7-9647-a556ed286159 admin admin] PATH_INFO: `/v3/users/0b9c17d96f3d44589cee145c8ec2e97d` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:39.652775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d424fe-15df-4ec7-9647-a556ed286159 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b9c17d96f3d44589cee145c8ec2e97d)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:10:39.654860 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d424fe-15df-4ec7-9647-a556ed286159 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:10:39.725342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b4d424fe-15df-4ec7-9647-a556ed286159 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b9c17d96f3d44589cee145c8ec2e97d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:10:39.726030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b4d424fe-15df-4ec7-9647-a556ed286159 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b9c17d96f3d44589cee145c8ec2e97d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:10:39.736895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b4d424fe-15df-4ec7-9647-a556ed286159 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b9c17d96f3d44589cee145c8ec2e97d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:10:39.745362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b4d424fe-15df-4ec7-9647-a556ed286159 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b9c17d96f3d44589cee145c8ec2e97d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:10:39.746165 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b4d424fe-15df-4ec7-9647-a556ed286159 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b9c17d96f3d44589cee145c8ec2e97d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:10:39.751465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:10:39.788857 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1148/2293] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:10:39 2022] DELETE /identity/v3/users/0b9c17d96f3d44589cee145c8ec2e97d => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:10:39.970010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cbe30-1b5a-41f2-a6b5-21b4e4ee085d service swift] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:10:39.998722 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cbe30-1b5a-41f2-a6b5-21b4e4ee085d service swift] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:10:40.069863 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78fe26-d9c1-4a73-9b06-0164d50a0289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:10:40.070551 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78fe26-d9c1-4a73-9b06-0164d50a0289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:40.070841 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78fe26-d9c1-4a73-9b06-0164d50a0289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:40.071254 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78fe26-d9c1-4a73-9b06-0164d50a0289 admin admin] PATH_INFO: `/v3/projects/0f8eb4de1820448eb1064b0fbb45a259` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:40.072422 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e78fe26-d9c1-4a73-9b06-0164d50a0289 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f8eb4de1820448eb1064b0fbb45a259)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:10:40.074354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e78fe26-d9c1-4a73-9b06-0164d50a0289 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:10:40.115482 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:10:40.131007 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1e78fe26-d9c1-4a73-9b06-0164d50a0289 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f8eb4de1820448eb1064b0fbb45a259', 'request_id': 'req-1e78fe26-d9c1-4a73-9b06-0164d50a0289'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:10:40.140452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1e78fe26-d9c1-4a73-9b06-0164d50a0289 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f8eb4de1820448eb1064b0fbb45a259', 'request_id': 'req-1e78fe26-d9c1-4a73-9b06-0164d50a0289'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:10:40.153935 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1e78fe26-d9c1-4a73-9b06-0164d50a0289 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f8eb4de1820448eb1064b0fbb45a259', 'request_id': 'req-1e78fe26-d9c1-4a73-9b06-0164d50a0289'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:10:40.167508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1e78fe26-d9c1-4a73-9b06-0164d50a0289 admin admin] The token cache is being invalidate because project 0f8eb4de1820448eb1064b0fbb45a259 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:10:40.167821 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1e78fe26-d9c1-4a73-9b06-0164d50a0289 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:10:40.168611 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-1e78fe26-d9c1-4a73-9b06-0164d50a0289 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:10:40.170108 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1146/2294] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:10:39 2022] DELETE /identity/v3/projects/0f8eb4de1820448eb1064b0fbb45a259 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:10:40.175121 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7c1eb-c68a-48df-ae78-68169c21b962 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:40.175411 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7c1eb-c68a-48df-ae78-68169c21b962 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:40.175740 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7c1eb-c68a-48df-ae78-68169c21b962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:40.195220 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-1ec7c1eb-c68a-48df-ae78-68169c21b962 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:10:40.196444 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1ec7c1eb-c68a-48df-ae78-68169c21b962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:10:40.278430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1149/2295] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:10:40.286657 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb16c71-745e-430b-ab77-23db2957dfde None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:40.286952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb16c71-745e-430b-ab77-23db2957dfde None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:40.287221 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb16c71-745e-430b-ab77-23db2957dfde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:40.307517 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-6eb16c71-745e-430b-ab77-23db2957dfde None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:10:40.308764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6eb16c71-745e-430b-ab77-23db2957dfde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:10:40.317708 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1147/2296] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:10:40.322129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7c1eb-c68a-48df-ae78-68169c21b962 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:10:40.333007 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7c1eb-c68a-48df-ae78-68169c21b962 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:10:40.340082 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f7f17-4770-42fd-b423-d941fdf050ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:10:40.341022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f7f17-4770-42fd-b423-d941fdf050ca admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:40.341342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f7f17-4770-42fd-b423-d941fdf050ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:40.341640 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f7f17-4770-42fd-b423-d941fdf050ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:40.342452 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f7f17-4770-42fd-b423-d941fdf050ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:10:40.342883 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f7f17-4770-42fd-b423-d941fdf050ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:10:40.344805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f7f17-4770-42fd-b423-d941fdf050ca admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:10:40.360766 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1150/2297] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:10:40.361859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb16c71-745e-430b-ab77-23db2957dfde None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:10:40.372857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb16c71-745e-430b-ab77-23db2957dfde None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:10:40.378619 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7a252-acd4-4ef0-b287-394b331f0a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:10:40.379655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7a252-acd4-4ef0-b287-394b331f0a5c admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:40.379940 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7a252-acd4-4ef0-b287-394b331f0a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:40.380215 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7a252-acd4-4ef0-b287-394b331f0a5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:40.380873 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7a252-acd4-4ef0-b287-394b331f0a5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:10:40.382843 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7a252-acd4-4ef0-b287-394b331f0a5c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:10:40.411791 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1148/2298] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:10:40 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:10:40.415933 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f7f17-4770-42fd-b423-d941fdf050ca admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:10:40.427195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f7f17-4770-42fd-b423-d941fdf050ca admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:10:40.492332 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b9388-3623-47d0-8b57-adf82d0532b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:10:40.492942 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b9388-3623-47d0-8b57-adf82d0532b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:40.493228 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b9388-3623-47d0-8b57-adf82d0532b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:40.493489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b9388-3623-47d0-8b57-adf82d0532b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:40.494113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b9388-3623-47d0-8b57-adf82d0532b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:10:40.496121 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b9388-3623-47d0-8b57-adf82d0532b7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:10:40.522162 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1151/2299] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:10:40 2022] POST /identity/v3/users => generated 300 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:10:40.526312 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7a252-acd4-4ef0-b287-394b331f0a5c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:10:40.538206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7a252-acd4-4ef0-b287-394b331f0a5c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:10:40.543191 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d0e25-99e9-452c-9570-3172a24d92df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:10:40.543839 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d0e25-99e9-452c-9570-3172a24d92df admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:40.544165 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d0e25-99e9-452c-9570-3172a24d92df admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:40.544482 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d0e25-99e9-452c-9570-3172a24d92df admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:40.545129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d0e25-99e9-452c-9570-3172a24d92df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:10:40.546807 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d0e25-99e9-452c-9570-3172a24d92df admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:10:40.558226 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1149/2300] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:10:40.562366 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b9388-3623-47d0-8b57-adf82d0532b7 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:10:40.572345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b9388-3623-47d0-8b57-adf82d0532b7 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:10:40.576935 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0b3b7-97e4-457c-a9db-1c3ee28f0828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:10:40.577531 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0b3b7-97e4-457c-a9db-1c3ee28f0828 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:40.577814 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0b3b7-97e4-457c-a9db-1c3ee28f0828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:40.578165 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0b3b7-97e4-457c-a9db-1c3ee28f0828 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:40.578797 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b0b3b7-97e4-457c-a9db-1c3ee28f0828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:10:40.580690 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b0b3b7-97e4-457c-a9db-1c3ee28f0828 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:10:40.592902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1152/2301] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:10:40.598106 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d0e25-99e9-452c-9570-3172a24d92df admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:10:40.608840 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d0e25-99e9-452c-9570-3172a24d92df admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:10:40.615162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155749fd-5e32-47e4-a069-c150bc6a0db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:10:40.616234 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155749fd-5e32-47e4-a069-c150bc6a0db0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:40.616519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155749fd-5e32-47e4-a069-c150bc6a0db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:40.616859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155749fd-5e32-47e4-a069-c150bc6a0db0 admin admin] PATH_INFO: `/v3/projects/febdbd894b0c4bc18265c6f4f84aa475/users/779c41b127094861a44e0a832baebaa6/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:40.633121 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155749fd-5e32-47e4-a069-c150bc6a0db0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=febdbd894b0c4bc18265c6f4f84aa475, user_id=779c41b127094861a44e0a832baebaa6, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:10:40.636077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155749fd-5e32-47e4-a069-c150bc6a0db0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:10:40.649121 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1150/2302] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:10:40 2022] PUT /identity/v3/projects/febdbd894b0c4bc18265c6f4f84aa475/users/779c41b127094861a44e0a832baebaa6/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:10:40.660046 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecb176-32f4-4c5f-aef1-b1ba0ad055d2 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:40.660334 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecb176-32f4-4c5f-aef1-b1ba0ad055d2 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:40.660609 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecb176-32f4-4c5f-aef1-b1ba0ad055d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:40.691028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-deecb176-32f4-4c5f-aef1-b1ba0ad055d2 None None] MFA Rules not processed for user `779c41b127094861a44e0a832baebaa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:10:40.692278 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-deecb176-32f4-4c5f-aef1-b1ba0ad055d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:10:40.738502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:10:40.766795 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1153/2303] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:10:40 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:10:40.781553 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155749fd-5e32-47e4-a069-c150bc6a0db0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:10:40.783740 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-155749fd-5e32-47e4-a069-c150bc6a0db0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:10:40.849402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155749fd-5e32-47e4-a069-c150bc6a0db0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:10:40.855597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066372e-9abc-4791-b251-6583e7112827 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:10:40.855597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066372e-9abc-4791-b251-6583e7112827 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:40.855926 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066372e-9abc-4791-b251-6583e7112827 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:40.856303 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066372e-9abc-4791-b251-6583e7112827 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:40.867670 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066372e-9abc-4791-b251-6583e7112827 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:10:40.869150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066372e-9abc-4791-b251-6583e7112827 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:10:40.897381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1151/2304] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:10:40 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:10:50.763173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=10 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:10:50.875868 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=63,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:10:51.094978 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecb176-32f4-4c5f-aef1-b1ba0ad055d2 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:10:51.097511 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-deecb176-32f4-4c5f-aef1-b1ba0ad055d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:10:51.176142 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecb176-32f4-4c5f-aef1-b1ba0ad055d2 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:10:51.181654 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fad95a-c11e-441c-8b1a-8a4d63fbdabc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:10:51.182254 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fad95a-c11e-441c-8b1a-8a4d63fbdabc service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:51.182533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fad95a-c11e-441c-8b1a-8a4d63fbdabc service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:51.182807 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fad95a-c11e-441c-8b1a-8a4d63fbdabc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:51.184532 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c2fad95a-c11e-441c-8b1a-8a4d63fbdabc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:10:51.244922 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fad95a-c11e-441c-8b1a-8a4d63fbdabc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:10:51.246526 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fad95a-c11e-441c-8b1a-8a4d63fbdabc service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:10:51.255817 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1154/2305] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:10:51 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:10:52.155939 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066372e-9abc-4791-b251-6583e7112827 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:10:52.220270 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066372e-9abc-4791-b251-6583e7112827 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:10:52.228239 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8228e8-dd29-49e3-87eb-e4523aa6f018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:10:52.229214 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8228e8-dd29-49e3-87eb-e4523aa6f018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:52.229740 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8228e8-dd29-49e3-87eb-e4523aa6f018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:52.230110 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8228e8-dd29-49e3-87eb-e4523aa6f018 admin admin] PATH_INFO: `/v3/users/779c41b127094861a44e0a832baebaa6` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:52.231446 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8228e8-dd29-49e3-87eb-e4523aa6f018 admin admin] RBAC: Authorizing `identity:delete_user(user_id=779c41b127094861a44e0a832baebaa6)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:10:52.233617 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8228e8-dd29-49e3-87eb-e4523aa6f018 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:10:52.293224 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fc8228e8-dd29-49e3-87eb-e4523aa6f018 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '779c41b127094861a44e0a832baebaa6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:10:52.293917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fc8228e8-dd29-49e3-87eb-e4523aa6f018 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '779c41b127094861a44e0a832baebaa6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:10:52.301433 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fc8228e8-dd29-49e3-87eb-e4523aa6f018 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '779c41b127094861a44e0a832baebaa6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:10:52.302135 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fc8228e8-dd29-49e3-87eb-e4523aa6f018 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '779c41b127094861a44e0a832baebaa6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:10:52.309538 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fc8228e8-dd29-49e3-87eb-e4523aa6f018 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '779c41b127094861a44e0a832baebaa6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:10:52.331111 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1152/2306] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:10:52 2022] DELETE /identity/v3/users/779c41b127094861a44e0a832baebaa6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:10:52.335447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ac73c-d7ce-4df9-a89d-faecf35cd6fe None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:52.335939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ac73c-d7ce-4df9-a89d-faecf35cd6fe None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:52.336167 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ac73c-d7ce-4df9-a89d-faecf35cd6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:52.355581 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-038ac73c-d7ce-4df9-a89d-faecf35cd6fe None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:10:52.356813 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-038ac73c-d7ce-4df9-a89d-faecf35cd6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:10:52.416021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1155/2307] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:10:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:10:52.425871 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8228e8-dd29-49e3-87eb-e4523aa6f018 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:10:52.427279 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fc8228e8-dd29-49e3-87eb-e4523aa6f018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:10:52.490197 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8228e8-dd29-49e3-87eb-e4523aa6f018 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:10:52.494329 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2781a9-a993-41e8-9c74-539534f9c243 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:10:52.494950 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2781a9-a993-41e8-9c74-539534f9c243 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:52.495231 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2781a9-a993-41e8-9c74-539534f9c243 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:52.495511 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2781a9-a993-41e8-9c74-539534f9c243 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:52.506734 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2781a9-a993-41e8-9c74-539534f9c243 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:10:52.508213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2781a9-a993-41e8-9c74-539534f9c243 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:10:52.519122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1153/2308] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:10:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:10:53.255663 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ac73c-d7ce-4df9-a89d-faecf35cd6fe None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:10:53.257095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-038ac73c-d7ce-4df9-a89d-faecf35cd6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:10:53.270292 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ac73c-d7ce-4df9-a89d-faecf35cd6fe None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:10:53.276176 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4067d90-9733-4d64-bffa-93d7e7d3eec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:10:53.276650 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4067d90-9733-4d64-bffa-93d7e7d3eec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:10:53.276997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4067d90-9733-4d64-bffa-93d7e7d3eec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:10:53.277334 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4067d90-9733-4d64-bffa-93d7e7d3eec1 admin admin] PATH_INFO: `/v3/projects/febdbd894b0c4bc18265c6f4f84aa475` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:10:53.278531 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4067d90-9733-4d64-bffa-93d7e7d3eec1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=febdbd894b0c4bc18265c6f4f84aa475)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:10:53.280654 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4067d90-9733-4d64-bffa-93d7e7d3eec1 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:10:53.321840 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:10:53.335951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b4067d90-9733-4d64-bffa-93d7e7d3eec1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'febdbd894b0c4bc18265c6f4f84aa475', 'request_id': 'req-b4067d90-9733-4d64-bffa-93d7e7d3eec1'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:10:53.344056 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b4067d90-9733-4d64-bffa-93d7e7d3eec1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'febdbd894b0c4bc18265c6f4f84aa475', 'request_id': 'req-b4067d90-9733-4d64-bffa-93d7e7d3eec1'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:10:53.350737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b4067d90-9733-4d64-bffa-93d7e7d3eec1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'febdbd894b0c4bc18265c6f4f84aa475', 'request_id': 'req-b4067d90-9733-4d64-bffa-93d7e7d3eec1'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:10:53.363001 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b4067d90-9733-4d64-bffa-93d7e7d3eec1 admin admin] The token cache is being invalidate because project febdbd894b0c4bc18265c6f4f84aa475 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:10:53.363422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b4067d90-9733-4d64-bffa-93d7e7d3eec1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:10:53.364125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-b4067d90-9733-4d64-bffa-93d7e7d3eec1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:10:53.365599 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1156/2309] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:10:53 2022] DELETE /identity/v3/projects/febdbd894b0c4bc18265c6f4f84aa475 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:11:00.324962 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2781a9-a993-41e8-9c74-539534f9c243 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:11:00.326416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5a2781a9-a993-41e8-9c74-539534f9c243 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:11:00.363215 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:11:00.423097 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2781a9-a993-41e8-9c74-539534f9c243 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:11:00.504640 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07daa9e-c50f-41c1-ad2a-1716e6b8230e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:11:00.505300 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07daa9e-c50f-41c1-ad2a-1716e6b8230e service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:11:00.505584 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07daa9e-c50f-41c1-ad2a-1716e6b8230e service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:11:00.505849 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07daa9e-c50f-41c1-ad2a-1716e6b8230e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:11:00.507591 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b07daa9e-c50f-41c1-ad2a-1716e6b8230e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:11:00.520966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07daa9e-c50f-41c1-ad2a-1716e6b8230e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:11:00.522564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07daa9e-c50f-41c1-ad2a-1716e6b8230e service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:11:00.533999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1154/2310] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:11:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:11:03.359598 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:11:03.580400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1a31c-d4a3-46a6-9178-cc2ac4328d1f None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:11:03.580688 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1a31c-d4a3-46a6-9178-cc2ac4328d1f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:11:03.580993 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1a31c-d4a3-46a6-9178-cc2ac4328d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:11:03.600620 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-dba1a31c-d4a3-46a6-9178-cc2ac4328d1f None None] MFA Rules not processed for user `ceccc909529042539dd2d5015058d8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:11:03.601810 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-dba1a31c-d4a3-46a6-9178-cc2ac4328d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:11:03.656484 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1157/2311] 10.222.0.42 () {58 vars in 967 bytes} [Wed Sep 14 01:11:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:11:03.679057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07daa9e-c50f-41c1-ad2a-1716e6b8230e service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:11:03.680472 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b07daa9e-c50f-41c1-ad2a-1716e6b8230e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:11:03.749311 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07daa9e-c50f-41c1-ad2a-1716e6b8230e service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:11:03.755345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0339824-b0c6-4b77-9bdd-78b142abed23 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e52776d9b4894af9b85a24136a8255e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:11:03.755977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0339824-b0c6-4b77-9bdd-78b142abed23 service swift] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:11:03.756256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0339824-b0c6-4b77-9bdd-78b142abed23 service swift] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:11:03.756530 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0339824-b0c6-4b77-9bdd-78b142abed23 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:11:03.765327 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0339824-b0c6-4b77-9bdd-78b142abed23 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:11:03.767258 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0339824-b0c6-4b77-9bdd-78b142abed23 service swift] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:11:03.775654 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1155/2312] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 01:11:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 01:11:13.649852 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=25 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:11:13.772734 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=84 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:11:21.674675 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ef7b8-0188-4af2-aaeb-a03ae634f67a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:11:21.675188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ef7b8-0188-4af2-aaeb-a03ae634f67a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:11:21.675292 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ef7b8-0188-4af2-aaeb-a03ae634f67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:11:21.694364 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-7d5ef7b8-0188-4af2-aaeb-a03ae634f67a None None] MFA Rules not processed for user `bf73eca572454b88843785d791f0b471`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:11:21.695642 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7d5ef7b8-0188-4af2-aaeb-a03ae634f67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:11:21.751657 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1158/2313] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:11:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:11:27.793866 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0339824-b0c6-4b77-9bdd-78b142abed23 service swift] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:11:27.795366 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f0339824-b0c6-4b77-9bdd-78b142abed23 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:11:27.813248 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0339824-b0c6-4b77-9bdd-78b142abed23 service swift] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:11:27.820720 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8873644-956f-46f5-a494-34290440f4c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:11:27.821295 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8873644-956f-46f5-a494-34290440f4c5 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:11:27.821587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8873644-956f-46f5-a494-34290440f4c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:11:27.821867 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8873644-956f-46f5-a494-34290440f4c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:11:27.823709 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a8873644-956f-46f5-a494-34290440f4c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:11:27.837896 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8873644-956f-46f5-a494-34290440f4c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:11:27.839497 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8873644-956f-46f5-a494-34290440f4c5 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:11:27.853705 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1156/2314] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:11:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:11:31.744788 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=25 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:11:37.848097 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=6 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:11:50.095903 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ef7b8-0188-4af2-aaeb-a03ae634f67a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:11:50.116753 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ef7b8-0188-4af2-aaeb-a03ae634f67a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:11:50.123438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba966d8-4a59-4a56-a6da-2434712a295a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:11:50.124471 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba966d8-4a59-4a56-a6da-2434712a295a service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:11:50.124926 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba966d8-4a59-4a56-a6da-2434712a295a service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:11:50.125381 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba966d8-4a59-4a56-a6da-2434712a295a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:11:50.127453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0ba966d8-4a59-4a56-a6da-2434712a295a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:11:50.144051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba966d8-4a59-4a56-a6da-2434712a295a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:11:50.146024 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba966d8-4a59-4a56-a6da-2434712a295a service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:11:50.162249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1159/2315] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:11:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:12:00.153391 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=6 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:12:02.555196 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334c199-3128-403c-b19d-2e32d538c280 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:02.555738 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334c199-3128-403c-b19d-2e32d538c280 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:02.555843 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334c199-3128-403c-b19d-2e32d538c280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:02.575134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4334c199-3128-403c-b19d-2e32d538c280 None None] MFA Rules not processed for user `51768890f19140b6819ef89f5bf610c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:12:02.576366 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4334c199-3128-403c-b19d-2e32d538c280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:12:02.629555 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1157/2316] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:12:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:12:02.956130 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba966d8-4a59-4a56-a6da-2434712a295a service nova] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:12:02.957543 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0ba966d8-4a59-4a56-a6da-2434712a295a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:12:03.016833 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba966d8-4a59-4a56-a6da-2434712a295a service nova] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:12:03.020413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857dd571-b050-4e9c-8c1f-81c6e4c527af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:12:03.021012 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dd571-b050-4e9c-8c1f-81c6e4c527af service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:03.021291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dd571-b050-4e9c-8c1f-81c6e4c527af service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:03.021564 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dd571-b050-4e9c-8c1f-81c6e4c527af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:03.023277 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-857dd571-b050-4e9c-8c1f-81c6e4c527af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:12:03.032885 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857dd571-b050-4e9c-8c1f-81c6e4c527af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:12:03.034344 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857dd571-b050-4e9c-8c1f-81c6e4c527af service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:12:03.042177 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1160/2317] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:12:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:12:12.622821 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=25 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:12:13.038704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=26 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:12:16.107862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23aa74-d6b1-4bbf-94d5-3b72ee73c478 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:16.108504 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23aa74-d6b1-4bbf-94d5-3b72ee73c478 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:16.108504 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23aa74-d6b1-4bbf-94d5-3b72ee73c478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:16.130057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0f23aa74-d6b1-4bbf-94d5-3b72ee73c478 None None] MFA Rules not processed for user `e4ad2d06f62246be8eb162257cbb4f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:12:16.131287 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0f23aa74-d6b1-4bbf-94d5-3b72ee73c478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:12:16.186443 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1158/2318] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:12:16 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:12:26.179867 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=25 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:12:48.015436 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857dd571-b050-4e9c-8c1f-81c6e4c527af service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:12:48.034598 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857dd571-b050-4e9c-8c1f-81c6e4c527af service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:12:48.038289 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a8938-c229-483b-9b93-7d391548db07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:12:48.038883 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a8938-c229-483b-9b93-7d391548db07 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:48.039160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a8938-c229-483b-9b93-7d391548db07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:48.039435 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a8938-c229-483b-9b93-7d391548db07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:48.041217 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-932a8938-c229-483b-9b93-7d391548db07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:12:48.155185 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a8938-c229-483b-9b93-7d391548db07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:12:48.156672 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a8938-c229-483b-9b93-7d391548db07 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:12:48.176177 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1161/2319] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:12:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:12:51.347187 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23aa74-d6b1-4bbf-94d5-3b72ee73c478 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:12:51.348826 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0f23aa74-d6b1-4bbf-94d5-3b72ee73c478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:12:51.446867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a8938-c229-483b-9b93-7d391548db07 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:12:51.449012 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-932a8938-c229-483b-9b93-7d391548db07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:12:51.465794 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a8938-c229-483b-9b93-7d391548db07 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:12:51.483455 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e036eac-e9d5-41e8-aa26-17faee58bbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:12:51.484646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e036eac-e9d5-41e8-aa26-17faee58bbe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:51.485176 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e036eac-e9d5-41e8-aa26-17faee58bbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:51.487467 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e036eac-e9d5-41e8-aa26-17faee58bbe9 admin admin] PATH_INFO: `/v3/users/bf73eca572454b88843785d791f0b471` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:51.489182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e036eac-e9d5-41e8-aa26-17faee58bbe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf73eca572454b88843785d791f0b471)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:12:51.491986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e036eac-e9d5-41e8-aa26-17faee58bbe9 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:12:51.558026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6e036eac-e9d5-41e8-aa26-17faee58bbe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf73eca572454b88843785d791f0b471'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:12:51.559465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6e036eac-e9d5-41e8-aa26-17faee58bbe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf73eca572454b88843785d791f0b471'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:12:51.561357 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23aa74-d6b1-4bbf-94d5-3b72ee73c478 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:12:51.567427 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb113c-858d-439d-a22d-e3302f18748b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18798838de284c84b5b838b3530a2c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:12:51.568089 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6e036eac-e9d5-41e8-aa26-17faee58bbe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf73eca572454b88843785d791f0b471'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:12:51.568220 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb113c-858d-439d-a22d-e3302f18748b service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:51.568381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb113c-858d-439d-a22d-e3302f18748b service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:51.568653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb113c-858d-439d-a22d-e3302f18748b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:51.570932 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ecbb113c-858d-439d-a22d-e3302f18748b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:12:51.575926 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6e036eac-e9d5-41e8-aa26-17faee58bbe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf73eca572454b88843785d791f0b471'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:12:51.576605 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6e036eac-e9d5-41e8-aa26-17faee58bbe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf73eca572454b88843785d791f0b471'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:12:51.590707 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:12:51.596372 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbb113c-858d-439d-a22d-e3302f18748b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:12:51.598157 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbb113c-858d-439d-a22d-e3302f18748b service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:12:51.598532 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1162/2320] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:12:51 2022] DELETE /identity/v3/users/bf73eca572454b88843785d791f0b471 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:12:51.610789 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1159/2321] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:12:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:12:51.784322 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e036eac-e9d5-41e8-aa26-17faee58bbe9 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:12:51.785825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6e036eac-e9d5-41e8-aa26-17faee58bbe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:12:51.857305 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e036eac-e9d5-41e8-aa26-17faee58bbe9 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:12:51.862240 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cadaf46-4c2d-4961-bf82-fc3f2c91dcdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:12:51.862967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cadaf46-4c2d-4961-bf82-fc3f2c91dcdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:51.863252 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cadaf46-4c2d-4961-bf82-fc3f2c91dcdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:51.863527 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cadaf46-4c2d-4961-bf82-fc3f2c91dcdf admin admin] PATH_INFO: `/v3/projects/d1dd868f069a46dbb6bfd549f6bbf7b8` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:51.864710 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cadaf46-4c2d-4961-bf82-fc3f2c91dcdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1dd868f069a46dbb6bfd549f6bbf7b8)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:12:51.866643 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cadaf46-4c2d-4961-bf82-fc3f2c91dcdf admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:12:51.919917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4cadaf46-4c2d-4961-bf82-fc3f2c91dcdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1dd868f069a46dbb6bfd549f6bbf7b8', 'request_id': 'req-4cadaf46-4c2d-4961-bf82-fc3f2c91dcdf'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:12:51.928395 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4cadaf46-4c2d-4961-bf82-fc3f2c91dcdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1dd868f069a46dbb6bfd549f6bbf7b8', 'request_id': 'req-4cadaf46-4c2d-4961-bf82-fc3f2c91dcdf'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:12:51.934287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4cadaf46-4c2d-4961-bf82-fc3f2c91dcdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1dd868f069a46dbb6bfd549f6bbf7b8', 'request_id': 'req-4cadaf46-4c2d-4961-bf82-fc3f2c91dcdf'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:12:51.946290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4cadaf46-4c2d-4961-bf82-fc3f2c91dcdf admin admin] The token cache is being invalidate because project d1dd868f069a46dbb6bfd549f6bbf7b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:12:51.946583 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4cadaf46-4c2d-4961-bf82-fc3f2c91dcdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:12:51.947239 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4cadaf46-4c2d-4961-bf82-fc3f2c91dcdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:12:51.948757 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1163/2322] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:12:51 2022] DELETE /identity/v3/projects/d1dd868f069a46dbb6bfd549f6bbf7b8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:12:51.953901 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c3d40-ff69-4e96-9964-0876e25a4c57 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:51.954191 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c3d40-ff69-4e96-9964-0876e25a4c57 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:51.954466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c3d40-ff69-4e96-9964-0876e25a4c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:52.017638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e98c3d40-ff69-4e96-9964-0876e25a4c57 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:12:52.018970 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e98c3d40-ff69-4e96-9964-0876e25a4c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:12:52.080164 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1160/2323] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:12:52.088144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeddf95-c3bc-47a7-a8c5-ff9090cd0a5b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:52.088436 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeddf95-c3bc-47a7-a8c5-ff9090cd0a5b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:52.088716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeddf95-c3bc-47a7-a8c5-ff9090cd0a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:52.108890 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-1eeddf95-c3bc-47a7-a8c5-ff9090cd0a5b None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:12:52.110099 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1eeddf95-c3bc-47a7-a8c5-ff9090cd0a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:12:52.118086 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1164/2324] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:12:52.122355 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c3d40-ff69-4e96-9964-0876e25a4c57 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:12:52.128615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:12:52.132696 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c3d40-ff69-4e96-9964-0876e25a4c57 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:12:52.138164 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e98b1c-cc5c-4e14-a4ad-fd8673925408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:12:52.138770 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e98b1c-cc5c-4e14-a4ad-fd8673925408 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:52.139054 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e98b1c-cc5c-4e14-a4ad-fd8673925408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:52.139328 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e98b1c-cc5c-4e14-a4ad-fd8673925408 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:52.140019 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e98b1c-cc5c-4e14-a4ad-fd8673925408 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:12:52.140327 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e98b1c-cc5c-4e14-a4ad-fd8673925408 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:12:52.141954 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e98b1c-cc5c-4e14-a4ad-fd8673925408 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:12:52.156776 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1161/2325] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:12:52.159946 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeddf95-c3bc-47a7-a8c5-ff9090cd0a5b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:12:52.169153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeddf95-c3bc-47a7-a8c5-ff9090cd0a5b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:12:52.173890 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b675c895-ab93-4469-af1f-3f463dee519c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:12:52.174487 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675c895-ab93-4469-af1f-3f463dee519c admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:52.174769 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675c895-ab93-4469-af1f-3f463dee519c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:52.175039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675c895-ab93-4469-af1f-3f463dee519c admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:52.175701 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675c895-ab93-4469-af1f-3f463dee519c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:12:52.177617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675c895-ab93-4469-af1f-3f463dee519c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:12:52.206742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1165/2326] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:12:52 2022] POST /identity/v3/projects => generated 357 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:12:52.210932 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e98b1c-cc5c-4e14-a4ad-fd8673925408 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:12:52.221350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e98b1c-cc5c-4e14-a4ad-fd8673925408 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:12:52.286985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219413b-71eb-4e81-8816-e57e54c1a141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:12:52.287518 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219413b-71eb-4e81-8816-e57e54c1a141 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:52.287881 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219413b-71eb-4e81-8816-e57e54c1a141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:52.288153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219413b-71eb-4e81-8816-e57e54c1a141 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:52.288794 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219413b-71eb-4e81-8816-e57e54c1a141 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:12:52.290698 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219413b-71eb-4e81-8816-e57e54c1a141 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:12:52.312651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1162/2327] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:12:52 2022] POST /identity/v3/users => generated 279 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:12:52.317598 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b675c895-ab93-4469-af1f-3f463dee519c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:12:52.328607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b675c895-ab93-4469-af1f-3f463dee519c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:12:52.333979 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b79c49-f3ac-4ea5-aa96-519d12d7f3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:12:52.334580 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b79c49-f3ac-4ea5-aa96-519d12d7f3ad admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:52.334921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b79c49-f3ac-4ea5-aa96-519d12d7f3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:52.335257 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b79c49-f3ac-4ea5-aa96-519d12d7f3ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:52.335929 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b79c49-f3ac-4ea5-aa96-519d12d7f3ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:12:52.337534 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b79c49-f3ac-4ea5-aa96-519d12d7f3ad admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:12:52.348028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1166/2328] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:12:52.352497 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219413b-71eb-4e81-8816-e57e54c1a141 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:12:52.363319 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219413b-71eb-4e81-8816-e57e54c1a141 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:12:52.368109 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd217d-fb30-45d6-8522-a291a7155388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:12:52.368723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd217d-fb30-45d6-8522-a291a7155388 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:52.369008 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd217d-fb30-45d6-8522-a291a7155388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:52.369288 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd217d-fb30-45d6-8522-a291a7155388 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:52.369933 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd217d-fb30-45d6-8522-a291a7155388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:12:52.371538 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd217d-fb30-45d6-8522-a291a7155388 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:12:52.382948 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1163/2329] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:12:52.387540 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b79c49-f3ac-4ea5-aa96-519d12d7f3ad admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:12:52.398261 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b79c49-f3ac-4ea5-aa96-519d12d7f3ad admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:12:52.402919 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd433ab7-9dc5-4100-a863-ed650056f413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:12:52.403602 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd433ab7-9dc5-4100-a863-ed650056f413 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:52.403874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd433ab7-9dc5-4100-a863-ed650056f413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:52.404151 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd433ab7-9dc5-4100-a863-ed650056f413 admin admin] PATH_INFO: `/v3/projects/324f2eb6c60f4c8998fbbd00ff3d4e27/users/73b8d9b0e6334ffa8704f132d79e1cf2/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:52.418975 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd433ab7-9dc5-4100-a863-ed650056f413 admin admin] RBAC: Authorizing `identity:create_grant(project_id=324f2eb6c60f4c8998fbbd00ff3d4e27, user_id=73b8d9b0e6334ffa8704f132d79e1cf2, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:12:52.421669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd433ab7-9dc5-4100-a863-ed650056f413 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:12:52.435052 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1167/2330] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:12:52 2022] PUT /identity/v3/projects/324f2eb6c60f4c8998fbbd00ff3d4e27/users/73b8d9b0e6334ffa8704f132d79e1cf2/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:12:52.447013 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def00253-e537-40c1-8e4a-e4352476561a None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:52.447384 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def00253-e537-40c1-8e4a-e4352476561a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:52.447765 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def00253-e537-40c1-8e4a-e4352476561a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:52.484701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-def00253-e537-40c1-8e4a-e4352476561a None None] MFA Rules not processed for user `73b8d9b0e6334ffa8704f132d79e1cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:12:52.486439 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-def00253-e537-40c1-8e4a-e4352476561a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:12:52.567595 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1164/2331] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:12:52 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:12:52.572458 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd433ab7-9dc5-4100-a863-ed650056f413 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:12:52.628009 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:12:52.694467 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd433ab7-9dc5-4100-a863-ed650056f413 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:12:52.700432 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5e812-bef3-4762-bd00-3f08b41ded40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:12:52.701959 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5e812-bef3-4762-bd00-3f08b41ded40 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:52.703088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5e812-bef3-4762-bd00-3f08b41ded40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:52.707956 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5e812-bef3-4762-bd00-3f08b41ded40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:52.708981 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d5e812-bef3-4762-bd00-3f08b41ded40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:12:52.713271 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d5e812-bef3-4762-bd00-3f08b41ded40 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:12:52.753255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1168/2332] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:12:52 2022] POST /identity/v3/projects => generated 357 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:12:52.754721 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def00253-e537-40c1-8e4a-e4352476561a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:12:52.767908 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def00253-e537-40c1-8e4a-e4352476561a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:12:52.774417 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5e812-bef3-4762-bd00-3f08b41ded40 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:12:52.777053 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-58d5e812-bef3-4762-bd00-3f08b41ded40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:12:52.851899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4378d-992a-4f4d-a661-a12b25d13c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:12:52.852871 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4378d-992a-4f4d-a661-a12b25d13c4a admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:52.853316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4378d-992a-4f4d-a661-a12b25d13c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:52.853600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4378d-992a-4f4d-a661-a12b25d13c4a admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:52.854246 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4378d-992a-4f4d-a661-a12b25d13c4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:12:52.856218 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4378d-992a-4f4d-a661-a12b25d13c4a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:12:52.864853 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:12:52.879877 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5e812-bef3-4762-bd00-3f08b41ded40 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:12:52.885394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c23ea-b1f9-4e99-8a8e-7a67816785ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18798838de284c84b5b838b3530a2c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:12:52.886160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c23ea-b1f9-4e99-8a8e-7a67816785ce service placement] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:52.886274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1165/2333] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:12:52 2022] POST /identity/v3/users => generated 285 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:12:52.886381 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c23ea-b1f9-4e99-8a8e-7a67816785ce service placement] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:52.886555 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c23ea-b1f9-4e99-8a8e-7a67816785ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:52.888489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-857c23ea-b1f9-4e99-8a8e-7a67816785ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:12:52.890075 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4378d-992a-4f4d-a661-a12b25d13c4a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:12:52.901131 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4378d-992a-4f4d-a661-a12b25d13c4a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:12:52.902275 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c23ea-b1f9-4e99-8a8e-7a67816785ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:12:52.904007 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c23ea-b1f9-4e99-8a8e-7a67816785ce service placement] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:12:52.906672 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4eb5ca-0518-4dbb-aa1e-fc33264f97e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:12:52.907409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4eb5ca-0518-4dbb-aa1e-fc33264f97e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:52.907897 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4eb5ca-0518-4dbb-aa1e-fc33264f97e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:52.908463 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4eb5ca-0518-4dbb-aa1e-fc33264f97e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:52.909241 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4eb5ca-0518-4dbb-aa1e-fc33264f97e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:12:52.910845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4eb5ca-0518-4dbb-aa1e-fc33264f97e7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:12:52.914520 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1169/2334] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:12:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:12:52.927108 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1166/2335] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:12:52.929372 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c23ea-b1f9-4e99-8a8e-7a67816785ce service placement] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:12:52.945761 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c23ea-b1f9-4e99-8a8e-7a67816785ce service placement] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:12:52.950360 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d507f4-10c3-4365-9754-b6a4709f89e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:12:52.951017 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d507f4-10c3-4365-9754-b6a4709f89e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:52.951294 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d507f4-10c3-4365-9754-b6a4709f89e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:52.951590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d507f4-10c3-4365-9754-b6a4709f89e7 admin admin] PATH_INFO: `/v3/projects/dec5433f33fb4a17a6198f1db1af3c72/users/98ca57f317d941629c4bae2a60814fda/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:52.967776 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d507f4-10c3-4365-9754-b6a4709f89e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dec5433f33fb4a17a6198f1db1af3c72, user_id=98ca57f317d941629c4bae2a60814fda, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:12:52.971277 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d507f4-10c3-4365-9754-b6a4709f89e7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:12:52.987659 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1170/2336] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:12:52 2022] PUT /identity/v3/projects/dec5433f33fb4a17a6198f1db1af3c72/users/98ca57f317d941629c4bae2a60814fda/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:12:52.991873 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4eb5ca-0518-4dbb-aa1e-fc33264f97e7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:12:53.002879 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4eb5ca-0518-4dbb-aa1e-fc33264f97e7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:12:53.071640 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d779e-de09-4b05-afb9-ea69c5f9e892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:12:53.072263 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d779e-de09-4b05-afb9-ea69c5f9e892 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:53.072544 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d779e-de09-4b05-afb9-ea69c5f9e892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:53.072825 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d779e-de09-4b05-afb9-ea69c5f9e892 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:53.073453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d779e-de09-4b05-afb9-ea69c5f9e892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:12:53.075798 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d779e-de09-4b05-afb9-ea69c5f9e892 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:12:53.090036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1167/2337] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:12:53.091241 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d507f4-10c3-4365-9754-b6a4709f89e7 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:12:53.105342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d507f4-10c3-4365-9754-b6a4709f89e7 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:12:53.111824 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf03a3b-cc12-4ec0-811e-fcfce2566833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:12:53.112489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf03a3b-cc12-4ec0-811e-fcfce2566833 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:53.113269 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf03a3b-cc12-4ec0-811e-fcfce2566833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:53.113548 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf03a3b-cc12-4ec0-811e-fcfce2566833 admin admin] PATH_INFO: `/v3/projects/dec5433f33fb4a17a6198f1db1af3c72/users/98ca57f317d941629c4bae2a60814fda/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:53.115486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf03a3b-cc12-4ec0-811e-fcfce2566833 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dec5433f33fb4a17a6198f1db1af3c72, user_id=98ca57f317d941629c4bae2a60814fda, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:12:53.118598 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf03a3b-cc12-4ec0-811e-fcfce2566833 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:12:53.130854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1171/2338] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:12:53 2022] PUT /identity/v3/projects/dec5433f33fb4a17a6198f1db1af3c72/users/98ca57f317d941629c4bae2a60814fda/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:12:53.141347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eadc93-57e5-449f-87fe-bf3b7746585f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:53.141785 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eadc93-57e5-449f-87fe-bf3b7746585f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:53.141925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eadc93-57e5-449f-87fe-bf3b7746585f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:53.178746 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d1eadc93-57e5-449f-87fe-bf3b7746585f None None] MFA Rules not processed for user `98ca57f317d941629c4bae2a60814fda`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:12:53.180038 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d1eadc93-57e5-449f-87fe-bf3b7746585f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:12:53.252850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1168/2339] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:12:53 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:12:53.264296 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf03a3b-cc12-4ec0-811e-fcfce2566833 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:12:53.266033 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ecf03a3b-cc12-4ec0-811e-fcfce2566833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:12:53.326710 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf03a3b-cc12-4ec0-811e-fcfce2566833 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:12:53.330885 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a292100-688e-4d17-8524-f35c02249eeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:12:53.331502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a292100-688e-4d17-8524-f35c02249eeb service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:53.331817 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a292100-688e-4d17-8524-f35c02249eeb service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:53.332092 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a292100-688e-4d17-8524-f35c02249eeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:53.341432 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a292100-688e-4d17-8524-f35c02249eeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:12:53.342868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a292100-688e-4d17-8524-f35c02249eeb service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:12:53.354003 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1172/2340] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:12:53 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:12:53.435030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eadc93-57e5-449f-87fe-bf3b7746585f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:12:53.446158 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eadc93-57e5-449f-87fe-bf3b7746585f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:12:53.449771 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf9c71-d025-4835-9a9c-fee46657770a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:12:53.450370 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf9c71-d025-4835-9a9c-fee46657770a service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:53.450646 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf9c71-d025-4835-9a9c-fee46657770a service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:53.450942 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf9c71-d025-4835-9a9c-fee46657770a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:53.513745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf9c71-d025-4835-9a9c-fee46657770a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:12:53.515632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf9c71-d025-4835-9a9c-fee46657770a service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:12:53.519683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:12:53.526435 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1169/2341] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:12:53 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:12:56.842733 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d4366-774e-43af-859e-951b1d010c95 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:12:56.843239 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d4366-774e-43af-859e-951b1d010c95 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:12:56.843353 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d4366-774e-43af-859e-951b1d010c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:12:56.852199 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:12:56.865799 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-f08d4366-774e-43af-859e-951b1d010c95 None None] MFA Rules not processed for user `2eca12e1e0a64eeeb183a328f338f54d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:12:56.866993 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f08d4366-774e-43af-859e-951b1d010c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:12:56.921414 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1173/2342] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:12:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:13:03.533960 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:13:06.914763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=23 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:13:08.574844 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf9c71-d025-4835-9a9c-fee46657770a service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:08.642791 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf9c71-d025-4835-9a9c-fee46657770a service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:08.647477 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0cbbf-99a7-48a5-b123-e75af7e96fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:08.648145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0cbbf-99a7-48a5-b123-e75af7e96fd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:08.648428 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0cbbf-99a7-48a5-b123-e75af7e96fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:08.648703 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0cbbf-99a7-48a5-b123-e75af7e96fd7 admin admin] PATH_INFO: `/v3/users/73b8d9b0e6334ffa8704f132d79e1cf2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:08.649823 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc0cbbf-99a7-48a5-b123-e75af7e96fd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73b8d9b0e6334ffa8704f132d79e1cf2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:08.651788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc0cbbf-99a7-48a5-b123-e75af7e96fd7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:08.702888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dfc0cbbf-99a7-48a5-b123-e75af7e96fd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73b8d9b0e6334ffa8704f132d79e1cf2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:08.703590 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dfc0cbbf-99a7-48a5-b123-e75af7e96fd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73b8d9b0e6334ffa8704f132d79e1cf2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:08.712966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dfc0cbbf-99a7-48a5-b123-e75af7e96fd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73b8d9b0e6334ffa8704f132d79e1cf2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:08.713591 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dfc0cbbf-99a7-48a5-b123-e75af7e96fd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73b8d9b0e6334ffa8704f132d79e1cf2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:08.720784 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dfc0cbbf-99a7-48a5-b123-e75af7e96fd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73b8d9b0e6334ffa8704f132d79e1cf2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:08.737699 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1170/2343] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:13:08 2022] DELETE /identity/v3/users/73b8d9b0e6334ffa8704f132d79e1cf2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:13:08.742124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4e933-4b5c-4191-a808-244200ad084d None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:08.742579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4e933-4b5c-4191-a808-244200ad084d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:08.742694 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4e933-4b5c-4191-a808-244200ad084d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:08.762159 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-1aa4e933-4b5c-4191-a808-244200ad084d None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:13:08.763358 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1aa4e933-4b5c-4191-a808-244200ad084d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:08.822904 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1174/2344] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:13:08.833481 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0cbbf-99a7-48a5-b123-e75af7e96fd7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:08.834916 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dfc0cbbf-99a7-48a5-b123-e75af7e96fd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:08.917894 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0cbbf-99a7-48a5-b123-e75af7e96fd7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:08.918125 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c00d1-cc8a-42bc-97d0-32343f1f766d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:08.918747 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c00d1-cc8a-42bc-97d0-32343f1f766d service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:08.919032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c00d1-cc8a-42bc-97d0-32343f1f766d service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:08.919306 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c00d1-cc8a-42bc-97d0-32343f1f766d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:08.929568 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c00d1-cc8a-42bc-97d0-32343f1f766d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:08.930999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c00d1-cc8a-42bc-97d0-32343f1f766d service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:08.940889 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1171/2345] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:13:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:13:09.216088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4e933-4b5c-4191-a808-244200ad084d None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:09.217610 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1aa4e933-4b5c-4191-a808-244200ad084d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:09.231374 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4e933-4b5c-4191-a808-244200ad084d None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:09.236751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97ad2e-b12e-4b10-a80f-11b5355bc4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:09.237385 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97ad2e-b12e-4b10-a80f-11b5355bc4df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:09.237674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97ad2e-b12e-4b10-a80f-11b5355bc4df admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:09.237950 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97ad2e-b12e-4b10-a80f-11b5355bc4df admin admin] PATH_INFO: `/v3/projects/324f2eb6c60f4c8998fbbd00ff3d4e27` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:09.239093 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97ad2e-b12e-4b10-a80f-11b5355bc4df admin admin] RBAC: Authorizing `identity:delete_project(project_id=324f2eb6c60f4c8998fbbd00ff3d4e27)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:09.241054 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97ad2e-b12e-4b10-a80f-11b5355bc4df admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:09.289361 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ac97ad2e-b12e-4b10-a80f-11b5355bc4df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '324f2eb6c60f4c8998fbbd00ff3d4e27', 'request_id': 'req-ac97ad2e-b12e-4b10-a80f-11b5355bc4df'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:09.296968 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ac97ad2e-b12e-4b10-a80f-11b5355bc4df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '324f2eb6c60f4c8998fbbd00ff3d4e27', 'request_id': 'req-ac97ad2e-b12e-4b10-a80f-11b5355bc4df'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:09.303262 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ac97ad2e-b12e-4b10-a80f-11b5355bc4df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '324f2eb6c60f4c8998fbbd00ff3d4e27', 'request_id': 'req-ac97ad2e-b12e-4b10-a80f-11b5355bc4df'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:09.315315 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ac97ad2e-b12e-4b10-a80f-11b5355bc4df admin admin] The token cache is being invalidate because project 324f2eb6c60f4c8998fbbd00ff3d4e27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:13:09.315658 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ac97ad2e-b12e-4b10-a80f-11b5355bc4df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:09.316293 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ac97ad2e-b12e-4b10-a80f-11b5355bc4df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:09.317773 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1175/2346] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:13:09 2022] DELETE /identity/v3/projects/324f2eb6c60f4c8998fbbd00ff3d4e27 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:13:09.321793 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c00d1-cc8a-42bc-97d0-32343f1f766d service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:09.326676 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:13:09.333387 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c00d1-cc8a-42bc-97d0-32343f1f766d service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:09.392662 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb28865-504d-4dac-b596-3fc1c10dc972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:09.393298 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb28865-504d-4dac-b596-3fc1c10dc972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:09.393578 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb28865-504d-4dac-b596-3fc1c10dc972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:09.393879 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb28865-504d-4dac-b596-3fc1c10dc972 admin admin] PATH_INFO: `/v3/users/98ca57f317d941629c4bae2a60814fda` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:09.395041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb28865-504d-4dac-b596-3fc1c10dc972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98ca57f317d941629c4bae2a60814fda)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:09.397204 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb28865-504d-4dac-b596-3fc1c10dc972 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:09.442240 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5fb28865-504d-4dac-b596-3fc1c10dc972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98ca57f317d941629c4bae2a60814fda'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:09.442889 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5fb28865-504d-4dac-b596-3fc1c10dc972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98ca57f317d941629c4bae2a60814fda'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:09.450280 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5fb28865-504d-4dac-b596-3fc1c10dc972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98ca57f317d941629c4bae2a60814fda'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:09.450900 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5fb28865-504d-4dac-b596-3fc1c10dc972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98ca57f317d941629c4bae2a60814fda'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:09.457988 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5fb28865-504d-4dac-b596-3fc1c10dc972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98ca57f317d941629c4bae2a60814fda'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:09.476273 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1172/2347] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:13:09 2022] DELETE /identity/v3/users/98ca57f317d941629c4bae2a60814fda => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:13:09.749267 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97ad2e-b12e-4b10-a80f-11b5355bc4df admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:09.751397 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ac97ad2e-b12e-4b10-a80f-11b5355bc4df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:09.770063 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97ad2e-b12e-4b10-a80f-11b5355bc4df admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:09.837311 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3ab93-ef6b-4774-8b93-019a4e4571aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:09.837955 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3ab93-ef6b-4774-8b93-019a4e4571aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:09.838245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3ab93-ef6b-4774-8b93-019a4e4571aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:09.838521 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3ab93-ef6b-4774-8b93-019a4e4571aa admin admin] PATH_INFO: `/v3/projects/dec5433f33fb4a17a6198f1db1af3c72` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:09.839678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3ab93-ef6b-4774-8b93-019a4e4571aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=dec5433f33fb4a17a6198f1db1af3c72)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:09.841607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3ab93-ef6b-4774-8b93-019a4e4571aa admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:09.874122 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:13:09.893781 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-37d3ab93-ef6b-4774-8b93-019a4e4571aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dec5433f33fb4a17a6198f1db1af3c72', 'request_id': 'req-37d3ab93-ef6b-4774-8b93-019a4e4571aa'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:09.902435 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-37d3ab93-ef6b-4774-8b93-019a4e4571aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dec5433f33fb4a17a6198f1db1af3c72', 'request_id': 'req-37d3ab93-ef6b-4774-8b93-019a4e4571aa'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:09.909267 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-37d3ab93-ef6b-4774-8b93-019a4e4571aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dec5433f33fb4a17a6198f1db1af3c72', 'request_id': 'req-37d3ab93-ef6b-4774-8b93-019a4e4571aa'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:09.921337 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-37d3ab93-ef6b-4774-8b93-019a4e4571aa admin admin] The token cache is being invalidate because project dec5433f33fb4a17a6198f1db1af3c72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:13:09.921635 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-37d3ab93-ef6b-4774-8b93-019a4e4571aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:09.922348 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-37d3ab93-ef6b-4774-8b93-019a4e4571aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:09.924510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1176/2348] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:13:09 2022] DELETE /identity/v3/projects/dec5433f33fb4a17a6198f1db1af3c72 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:13:09.930488 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074feb4d-df14-4b1a-82f7-49e64e7d9c19 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:09.931005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074feb4d-df14-4b1a-82f7-49e64e7d9c19 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:09.931110 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074feb4d-df14-4b1a-82f7-49e64e7d9c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:09.951821 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-074feb4d-df14-4b1a-82f7-49e64e7d9c19 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:13:09.953068 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-074feb4d-df14-4b1a-82f7-49e64e7d9c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:10.018334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1173/2349] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:13:10.027521 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfee01a-a705-4533-b605-0b4ee93f8e86 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:10.027855 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfee01a-a705-4533-b605-0b4ee93f8e86 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:10.028137 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfee01a-a705-4533-b605-0b4ee93f8e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:10.048952 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-fdfee01a-a705-4533-b605-0b4ee93f8e86 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:13:10.050160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-fdfee01a-a705-4533-b605-0b4ee93f8e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:10.058231 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1177/2350] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:13:10.062226 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074feb4d-df14-4b1a-82f7-49e64e7d9c19 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:10.072370 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074feb4d-df14-4b1a-82f7-49e64e7d9c19 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:10.077220 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace63b41-de17-440e-b870-5c80c6268103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:10.077847 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace63b41-de17-440e-b870-5c80c6268103 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:10.078125 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace63b41-de17-440e-b870-5c80c6268103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:10.078395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace63b41-de17-440e-b870-5c80c6268103 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:10.079726 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace63b41-de17-440e-b870-5c80c6268103 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:13:10.080029 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace63b41-de17-440e-b870-5c80c6268103 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:10.081623 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace63b41-de17-440e-b870-5c80c6268103 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:10.093501 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1174/2351] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:13:10.098483 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfee01a-a705-4533-b605-0b4ee93f8e86 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:10.107936 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfee01a-a705-4533-b605-0b4ee93f8e86 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:10.113508 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237fff08-fad8-42b6-8088-47e5bdf11326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:10.114104 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237fff08-fad8-42b6-8088-47e5bdf11326 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:10.114387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237fff08-fad8-42b6-8088-47e5bdf11326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:10.114658 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237fff08-fad8-42b6-8088-47e5bdf11326 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:10.115287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237fff08-fad8-42b6-8088-47e5bdf11326 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:10.117211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237fff08-fad8-42b6-8088-47e5bdf11326 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:10.150763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1178/2352] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:13:10 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:13:10.155150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace63b41-de17-440e-b870-5c80c6268103 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:10.166083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace63b41-de17-440e-b870-5c80c6268103 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:10.171170 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:13:10.231034 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00a3af-fb40-4ae3-ba55-26bb1c5aeca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:10.231809 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00a3af-fb40-4ae3-ba55-26bb1c5aeca3 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:10.232123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00a3af-fb40-4ae3-ba55-26bb1c5aeca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:10.232396 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00a3af-fb40-4ae3-ba55-26bb1c5aeca3 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:10.233047 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd00a3af-fb40-4ae3-ba55-26bb1c5aeca3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:10.235598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd00a3af-fb40-4ae3-ba55-26bb1c5aeca3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:10.256945 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1175/2353] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:13:10 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:13:10.261667 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237fff08-fad8-42b6-8088-47e5bdf11326 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:10.272453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237fff08-fad8-42b6-8088-47e5bdf11326 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:10.278029 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914de73-66ec-4648-9e80-f0f8ac862825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:10.278638 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914de73-66ec-4648-9e80-f0f8ac862825 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:10.278921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914de73-66ec-4648-9e80-f0f8ac862825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:10.279197 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914de73-66ec-4648-9e80-f0f8ac862825 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:10.279881 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b914de73-66ec-4648-9e80-f0f8ac862825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:10.281479 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b914de73-66ec-4648-9e80-f0f8ac862825 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:10.292759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1179/2354] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:13:10.296709 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00a3af-fb40-4ae3-ba55-26bb1c5aeca3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:10.306775 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00a3af-fb40-4ae3-ba55-26bb1c5aeca3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:10.311435 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771736c9-d078-4ec5-8683-60789de5961c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:10.312076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771736c9-d078-4ec5-8683-60789de5961c admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:10.312376 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771736c9-d078-4ec5-8683-60789de5961c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:10.312635 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771736c9-d078-4ec5-8683-60789de5961c admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:10.313884 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771736c9-d078-4ec5-8683-60789de5961c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:10.315484 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771736c9-d078-4ec5-8683-60789de5961c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:10.326861 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1176/2355] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:13:10.331838 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914de73-66ec-4648-9e80-f0f8ac862825 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:10.342506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914de73-66ec-4648-9e80-f0f8ac862825 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:10.349224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7853bf4-56c4-4a70-94a8-a314eb542584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:10.350083 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7853bf4-56c4-4a70-94a8-a314eb542584 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:10.350414 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7853bf4-56c4-4a70-94a8-a314eb542584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:10.350692 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7853bf4-56c4-4a70-94a8-a314eb542584 admin admin] PATH_INFO: `/v3/projects/efc10efc11134c0d9e289317611f9cc1/users/2f4da1a552f04aa8abc5083d86681a02/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:10.366307 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7853bf4-56c4-4a70-94a8-a314eb542584 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc10efc11134c0d9e289317611f9cc1, user_id=2f4da1a552f04aa8abc5083d86681a02, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:10.369052 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7853bf4-56c4-4a70-94a8-a314eb542584 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:10.391809 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1180/2356] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:13:10 2022] PUT /identity/v3/projects/efc10efc11134c0d9e289317611f9cc1/users/2f4da1a552f04aa8abc5083d86681a02/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:13:10.397117 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e028d3-4ec3-4fbc-9f87-445b023c4efb None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:10.397408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e028d3-4ec3-4fbc-9f87-445b023c4efb None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:10.397805 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e028d3-4ec3-4fbc-9f87-445b023c4efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:10.426310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f8e028d3-4ec3-4fbc-9f87-445b023c4efb None None] MFA Rules not processed for user `2f4da1a552f04aa8abc5083d86681a02`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:13:10.427579 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f8e028d3-4ec3-4fbc-9f87-445b023c4efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:10.499588 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1177/2357] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:13:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:13:10.502571 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7853bf4-56c4-4a70-94a8-a314eb542584 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:10.565692 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7853bf4-56c4-4a70-94a8-a314eb542584 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:10.570185 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85897d89-923d-4d8b-8616-cd11a2bd5e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:10.570787 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85897d89-923d-4d8b-8616-cd11a2bd5e0b admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:10.571064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85897d89-923d-4d8b-8616-cd11a2bd5e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:10.571334 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85897d89-923d-4d8b-8616-cd11a2bd5e0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:10.572012 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85897d89-923d-4d8b-8616-cd11a2bd5e0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:10.573921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85897d89-923d-4d8b-8616-cd11a2bd5e0b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:10.601084 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1181/2358] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:13:10 2022] POST /identity/v3/projects => generated 389 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:13:10.605256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e028d3-4ec3-4fbc-9f87-445b023c4efb None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:10.616882 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e028d3-4ec3-4fbc-9f87-445b023c4efb None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:10.675668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97e58c-4ad3-4b56-a067-47e898f94136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:10.676281 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97e58c-4ad3-4b56-a067-47e898f94136 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:10.676560 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97e58c-4ad3-4b56-a067-47e898f94136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:10.676834 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97e58c-4ad3-4b56-a067-47e898f94136 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:10.677478 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97e58c-4ad3-4b56-a067-47e898f94136 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:10.679409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97e58c-4ad3-4b56-a067-47e898f94136 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:10.706715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1178/2359] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:13:10 2022] POST /identity/v3/users => generated 301 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:13:10.710665 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85897d89-923d-4d8b-8616-cd11a2bd5e0b admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:10.721964 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85897d89-923d-4d8b-8616-cd11a2bd5e0b admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:10.727333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d62ec-748f-4f12-b557-b38428de3c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:10.727973 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d62ec-748f-4f12-b557-b38428de3c53 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:10.728254 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d62ec-748f-4f12-b557-b38428de3c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:10.729304 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d62ec-748f-4f12-b557-b38428de3c53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:10.729810 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d62ec-748f-4f12-b557-b38428de3c53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:10.731494 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d62ec-748f-4f12-b557-b38428de3c53 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:10.743717 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1182/2360] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:13:10.748055 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97e58c-4ad3-4b56-a067-47e898f94136 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:10.759681 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97e58c-4ad3-4b56-a067-47e898f94136 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:10.764329 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66932bb-d716-4fc2-a97e-66bfbfb27500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:10.764985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66932bb-d716-4fc2-a97e-66bfbfb27500 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:10.765261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66932bb-d716-4fc2-a97e-66bfbfb27500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:10.765530 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66932bb-d716-4fc2-a97e-66bfbfb27500 admin admin] PATH_INFO: `/v3/projects/6b3707e0031a4095b0a74ed4548eafc0/users/b2faeeab90bd441ba05d6a9f468d29f3/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:10.768508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:13:10.785410 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66932bb-d716-4fc2-a97e-66bfbfb27500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b3707e0031a4095b0a74ed4548eafc0, user_id=b2faeeab90bd441ba05d6a9f468d29f3, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:10.788315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66932bb-d716-4fc2-a97e-66bfbfb27500 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:10.802222 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1179/2361] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:13:10 2022] PUT /identity/v3/projects/6b3707e0031a4095b0a74ed4548eafc0/users/b2faeeab90bd441ba05d6a9f468d29f3/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:13:10.806373 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d62ec-748f-4f12-b557-b38428de3c53 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:10.817754 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d62ec-748f-4f12-b557-b38428de3c53 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:10.838353 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:13:10.879096 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9c51c-b3b9-48a8-b5b4-eff52cc0f330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:10.879762 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9c51c-b3b9-48a8-b5b4-eff52cc0f330 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:10.880043 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9c51c-b3b9-48a8-b5b4-eff52cc0f330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:10.880314 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9c51c-b3b9-48a8-b5b4-eff52cc0f330 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:10.880976 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9c51c-b3b9-48a8-b5b4-eff52cc0f330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:10.882574 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9c51c-b3b9-48a8-b5b4-eff52cc0f330 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:10.893755 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1183/2362] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:13:10.897922 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66932bb-d716-4fc2-a97e-66bfbfb27500 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:10.908896 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66932bb-d716-4fc2-a97e-66bfbfb27500 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:10.913589 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf68a4-1794-4258-b202-299a233c93bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:10.914254 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf68a4-1794-4258-b202-299a233c93bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:10.914965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf68a4-1794-4258-b202-299a233c93bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:10.914965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf68a4-1794-4258-b202-299a233c93bd admin admin] PATH_INFO: `/v3/projects/6b3707e0031a4095b0a74ed4548eafc0/users/b2faeeab90bd441ba05d6a9f468d29f3/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:10.917016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf68a4-1794-4258-b202-299a233c93bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b3707e0031a4095b0a74ed4548eafc0, user_id=b2faeeab90bd441ba05d6a9f468d29f3, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:10.921138 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf68a4-1794-4258-b202-299a233c93bd admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:10.934617 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1180/2363] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:13:10 2022] PUT /identity/v3/projects/6b3707e0031a4095b0a74ed4548eafc0/users/b2faeeab90bd441ba05d6a9f468d29f3/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:13:10.945737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff52ef4-7ce0-486f-8a5d-01db574356a2 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:10.946105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff52ef4-7ce0-486f-8a5d-01db574356a2 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:10.946310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff52ef4-7ce0-486f-8a5d-01db574356a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:10.977095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-1ff52ef4-7ce0-486f-8a5d-01db574356a2 None None] MFA Rules not processed for user `b2faeeab90bd441ba05d6a9f468d29f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:13:10.978640 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1ff52ef4-7ce0-486f-8a5d-01db574356a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:11.073815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1184/2364] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:13:10 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:13:11.088708 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf68a4-1794-4258-b202-299a233c93bd admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:11.090479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7baf68a4-1794-4258-b202-299a233c93bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:11.186201 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf68a4-1794-4258-b202-299a233c93bd admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:11.192761 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa78da-0a78-4483-8143-b1c691c5887f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:11.192761 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa78da-0a78-4483-8143-b1c691c5887f service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:11.192761 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa78da-0a78-4483-8143-b1c691c5887f service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:11.192761 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa78da-0a78-4483-8143-b1c691c5887f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:11.202923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa78da-0a78-4483-8143-b1c691c5887f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:11.204512 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa78da-0a78-4483-8143-b1c691c5887f service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:11.214967 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1181/2365] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:13:11 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:13:11.363946 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff52ef4-7ce0-486f-8a5d-01db574356a2 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:11.435283 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff52ef4-7ce0-486f-8a5d-01db574356a2 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:11.440598 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4aa08d-b7d8-4f65-afe2-4b43d564ca70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:11.441289 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4aa08d-b7d8-4f65-afe2-4b43d564ca70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:11.441577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4aa08d-b7d8-4f65-afe2-4b43d564ca70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:11.441857 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4aa08d-b7d8-4f65-afe2-4b43d564ca70 admin admin] PATH_INFO: `/v3/users/2f4da1a552f04aa8abc5083d86681a02` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:11.443005 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4aa08d-b7d8-4f65-afe2-4b43d564ca70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f4da1a552f04aa8abc5083d86681a02)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:11.445134 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4aa08d-b7d8-4f65-afe2-4b43d564ca70 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:11.512186 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-bf4aa08d-b7d8-4f65-afe2-4b43d564ca70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f4da1a552f04aa8abc5083d86681a02'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:11.513235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-bf4aa08d-b7d8-4f65-afe2-4b43d564ca70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f4da1a552f04aa8abc5083d86681a02'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:11.515311 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:13:11.523375 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-bf4aa08d-b7d8-4f65-afe2-4b43d564ca70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f4da1a552f04aa8abc5083d86681a02'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:11.532275 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-bf4aa08d-b7d8-4f65-afe2-4b43d564ca70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f4da1a552f04aa8abc5083d86681a02'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:11.533128 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-bf4aa08d-b7d8-4f65-afe2-4b43d564ca70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f4da1a552f04aa8abc5083d86681a02'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:11.556357 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1185/2366] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:13:11 2022] DELETE /identity/v3/users/2f4da1a552f04aa8abc5083d86681a02 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:13:11.560912 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17068e2f-7a3a-4eee-8d6b-f7d1a1a11b66 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:11.561529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17068e2f-7a3a-4eee-8d6b-f7d1a1a11b66 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:11.562014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17068e2f-7a3a-4eee-8d6b-f7d1a1a11b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:11.584183 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-17068e2f-7a3a-4eee-8d6b-f7d1a1a11b66 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:13:11.585628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-17068e2f-7a3a-4eee-8d6b-f7d1a1a11b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:11.660594 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1182/2367] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:13:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:13:11.668921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4aa08d-b7d8-4f65-afe2-4b43d564ca70 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:11.670379 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bf4aa08d-b7d8-4f65-afe2-4b43d564ca70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:11.738156 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4aa08d-b7d8-4f65-afe2-4b43d564ca70 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:11.742065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599d351-cfd8-4e7a-8dde-316f43beebb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:11.742684 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599d351-cfd8-4e7a-8dde-316f43beebb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:11.742967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599d351-cfd8-4e7a-8dde-316f43beebb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:11.743241 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599d351-cfd8-4e7a-8dde-316f43beebb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:11.753458 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599d351-cfd8-4e7a-8dde-316f43beebb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:11.754914 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599d351-cfd8-4e7a-8dde-316f43beebb4 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:11.767174 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1186/2368] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:13:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:13:12.054196 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17068e2f-7a3a-4eee-8d6b-f7d1a1a11b66 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:12.055849 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-17068e2f-7a3a-4eee-8d6b-f7d1a1a11b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:12.068754 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17068e2f-7a3a-4eee-8d6b-f7d1a1a11b66 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:12.074265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a89be-9c7c-4d49-9541-723aa2c60a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:12.074926 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a89be-9c7c-4d49-9541-723aa2c60a71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:12.075209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a89be-9c7c-4d49-9541-723aa2c60a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:12.075487 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a89be-9c7c-4d49-9541-723aa2c60a71 admin admin] PATH_INFO: `/v3/projects/efc10efc11134c0d9e289317611f9cc1` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:12.076675 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a89be-9c7c-4d49-9541-723aa2c60a71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efc10efc11134c0d9e289317611f9cc1)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:12.079185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a89be-9c7c-4d49-9541-723aa2c60a71 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:12.130199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:13:12.132010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-908a89be-9c7c-4d49-9541-723aa2c60a71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efc10efc11134c0d9e289317611f9cc1', 'request_id': 'req-908a89be-9c7c-4d49-9541-723aa2c60a71'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:12.140312 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-908a89be-9c7c-4d49-9541-723aa2c60a71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efc10efc11134c0d9e289317611f9cc1', 'request_id': 'req-908a89be-9c7c-4d49-9541-723aa2c60a71'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:12.147065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-908a89be-9c7c-4d49-9541-723aa2c60a71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efc10efc11134c0d9e289317611f9cc1', 'request_id': 'req-908a89be-9c7c-4d49-9541-723aa2c60a71'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:12.158922 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-908a89be-9c7c-4d49-9541-723aa2c60a71 admin admin] The token cache is being invalidate because project efc10efc11134c0d9e289317611f9cc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:13:12.159294 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-908a89be-9c7c-4d49-9541-723aa2c60a71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:12.160053 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-908a89be-9c7c-4d49-9541-723aa2c60a71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:12.161548 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1183/2369] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:13:12 2022] DELETE /identity/v3/projects/efc10efc11134c0d9e289317611f9cc1 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:13:12.165699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599d351-cfd8-4e7a-8dde-316f43beebb4 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:12.175421 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599d351-cfd8-4e7a-8dde-316f43beebb4 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:12.237674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf16640-228b-416d-b395-767a5b622105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:12.238450 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf16640-228b-416d-b395-767a5b622105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:12.238733 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf16640-228b-416d-b395-767a5b622105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:12.239133 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf16640-228b-416d-b395-767a5b622105 admin admin] PATH_INFO: `/v3/users/b2faeeab90bd441ba05d6a9f468d29f3` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:12.240314 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf16640-228b-416d-b395-767a5b622105 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2faeeab90bd441ba05d6a9f468d29f3)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:12.242277 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf16640-228b-416d-b395-767a5b622105 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:12.290390 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6bf16640-228b-416d-b395-767a5b622105 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2faeeab90bd441ba05d6a9f468d29f3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:12.291032 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6bf16640-228b-416d-b395-767a5b622105 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2faeeab90bd441ba05d6a9f468d29f3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:12.298638 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6bf16640-228b-416d-b395-767a5b622105 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2faeeab90bd441ba05d6a9f468d29f3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:12.305663 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6bf16640-228b-416d-b395-767a5b622105 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2faeeab90bd441ba05d6a9f468d29f3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:12.306298 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6bf16640-228b-416d-b395-767a5b622105 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2faeeab90bd441ba05d6a9f468d29f3'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:12.311711 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:13:12.325734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1187/2370] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:13:12 2022] DELETE /identity/v3/users/b2faeeab90bd441ba05d6a9f468d29f3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:13:12.592979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a89be-9c7c-4d49-9541-723aa2c60a71 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:12.595277 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-908a89be-9c7c-4d49-9541-723aa2c60a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:12.608746 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a89be-9c7c-4d49-9541-723aa2c60a71 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:12.665424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad2095e-e713-48c9-99ae-ebb616b6e827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:12.666054 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2095e-e713-48c9-99ae-ebb616b6e827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:12.666335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2095e-e713-48c9-99ae-ebb616b6e827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:12.666610 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2095e-e713-48c9-99ae-ebb616b6e827 admin admin] PATH_INFO: `/v3/projects/6b3707e0031a4095b0a74ed4548eafc0` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:12.667749 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad2095e-e713-48c9-99ae-ebb616b6e827 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b3707e0031a4095b0a74ed4548eafc0)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:12.669672 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad2095e-e713-48c9-99ae-ebb616b6e827 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:12.715874 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fad2095e-e713-48c9-99ae-ebb616b6e827 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b3707e0031a4095b0a74ed4548eafc0', 'request_id': 'req-fad2095e-e713-48c9-99ae-ebb616b6e827'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:12.723358 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fad2095e-e713-48c9-99ae-ebb616b6e827 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b3707e0031a4095b0a74ed4548eafc0', 'request_id': 'req-fad2095e-e713-48c9-99ae-ebb616b6e827'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:12.729071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fad2095e-e713-48c9-99ae-ebb616b6e827 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b3707e0031a4095b0a74ed4548eafc0', 'request_id': 'req-fad2095e-e713-48c9-99ae-ebb616b6e827'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:12.739873 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fad2095e-e713-48c9-99ae-ebb616b6e827 admin admin] The token cache is being invalidate because project 6b3707e0031a4095b0a74ed4548eafc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:13:12.740167 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fad2095e-e713-48c9-99ae-ebb616b6e827 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:12.740791 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fad2095e-e713-48c9-99ae-ebb616b6e827 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:12.742282 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1184/2371] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:13:12 2022] DELETE /identity/v3/projects/6b3707e0031a4095b0a74ed4548eafc0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:13:12.747965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f961d8-3ab7-450e-94b5-c2977887c698 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:12.748256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f961d8-3ab7-450e-94b5-c2977887c698 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:12.748561 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f961d8-3ab7-450e-94b5-c2977887c698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:12.770795 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e9f961d8-3ab7-450e-94b5-c2977887c698 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:13:12.772105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e9f961d8-3ab7-450e-94b5-c2977887c698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:12.833456 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1188/2372] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:13:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:13:12.842369 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198e0da-fa44-4c34-9945-7708081dd85d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:12.842657 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198e0da-fa44-4c34-9945-7708081dd85d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:12.842932 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198e0da-fa44-4c34-9945-7708081dd85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:12.863428 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-5198e0da-fa44-4c34-9945-7708081dd85d None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:13:12.864582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5198e0da-fa44-4c34-9945-7708081dd85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:12.872412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1185/2373] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:13:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:13:12.877535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f961d8-3ab7-450e-94b5-c2977887c698 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:12.887613 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f961d8-3ab7-450e-94b5-c2977887c698 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:12.892108 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11067277-f9d8-456b-86c7-c2cae7761dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:12.892694 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11067277-f9d8-456b-86c7-c2cae7761dd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:12.892976 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11067277-f9d8-456b-86c7-c2cae7761dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:12.893245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11067277-f9d8-456b-86c7-c2cae7761dd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:12.893883 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11067277-f9d8-456b-86c7-c2cae7761dd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:13:12.894179 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11067277-f9d8-456b-86c7-c2cae7761dd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:12.895805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11067277-f9d8-456b-86c7-c2cae7761dd2 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:12.907082 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1189/2374] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:13:12.911302 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5198e0da-fa44-4c34-9945-7708081dd85d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:12.920454 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5198e0da-fa44-4c34-9945-7708081dd85d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:12.925883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d2fd3-a10e-44b7-80e2-9ec9f1a25ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:12.926478 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d2fd3-a10e-44b7-80e2-9ec9f1a25ce5 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:12.926759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d2fd3-a10e-44b7-80e2-9ec9f1a25ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:12.927033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d2fd3-a10e-44b7-80e2-9ec9f1a25ce5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:12.927702 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d2fd3-a10e-44b7-80e2-9ec9f1a25ce5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:12.929604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2d2fd3-a10e-44b7-80e2-9ec9f1a25ce5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:12.957513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1186/2375] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:13:12 2022] POST /identity/v3/projects => generated 359 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:13:12.961662 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11067277-f9d8-456b-86c7-c2cae7761dd2 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:12.971724 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11067277-f9d8-456b-86c7-c2cae7761dd2 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:13.028550 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df0f6d-4176-4c2b-ab1d-5451cd0cfb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:13.029282 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df0f6d-4176-4c2b-ab1d-5451cd0cfb81 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:13.029633 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df0f6d-4176-4c2b-ab1d-5451cd0cfb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:13.029979 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df0f6d-4176-4c2b-ab1d-5451cd0cfb81 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:13.030696 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df0f6d-4176-4c2b-ab1d-5451cd0cfb81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:13.032746 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df0f6d-4176-4c2b-ab1d-5451cd0cfb81 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:13.055110 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1190/2376] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:13:12 2022] POST /identity/v3/users => generated 280 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:13:13.059524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d2fd3-a10e-44b7-80e2-9ec9f1a25ce5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:13.071076 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d2fd3-a10e-44b7-80e2-9ec9f1a25ce5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:13.075920 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2549c3-aa55-4408-9b0e-bbd2d55534a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:13.076541 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2549c3-aa55-4408-9b0e-bbd2d55534a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:13.076816 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2549c3-aa55-4408-9b0e-bbd2d55534a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:13.077083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2549c3-aa55-4408-9b0e-bbd2d55534a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:13.077720 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2549c3-aa55-4408-9b0e-bbd2d55534a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:13.079706 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2549c3-aa55-4408-9b0e-bbd2d55534a4 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:13.090343 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1187/2377] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:13:13.096264 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df0f6d-4176-4c2b-ab1d-5451cd0cfb81 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:13.109023 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df0f6d-4176-4c2b-ab1d-5451cd0cfb81 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:13.115669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c573f4-eaf3-457f-849a-d48aefda4ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:13.116388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c573f4-eaf3-457f-849a-d48aefda4ffc admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:13.116388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c573f4-eaf3-457f-849a-d48aefda4ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:13.116583 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c573f4-eaf3-457f-849a-d48aefda4ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:13.117380 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c573f4-eaf3-457f-849a-d48aefda4ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:13.118984 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c573f4-eaf3-457f-849a-d48aefda4ffc admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:13.133613 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1191/2378] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:13:13.138064 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2549c3-aa55-4408-9b0e-bbd2d55534a4 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:13.148930 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2549c3-aa55-4408-9b0e-bbd2d55534a4 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:13.153604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f13ca-fe15-4828-a559-06c4c867ede2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:13.154260 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f13ca-fe15-4828-a559-06c4c867ede2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:13.154538 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f13ca-fe15-4828-a559-06c4c867ede2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:13.154815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f13ca-fe15-4828-a559-06c4c867ede2 admin admin] PATH_INFO: `/v3/projects/e76d5af3829b4fd5bc75fa2594e1ad6f/users/f3abd0b11a9f4e11b7bf95b536c90198/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:13.171568 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f13ca-fe15-4828-a559-06c4c867ede2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e76d5af3829b4fd5bc75fa2594e1ad6f, user_id=f3abd0b11a9f4e11b7bf95b536c90198, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:13.174784 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f13ca-fe15-4828-a559-06c4c867ede2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:13.188593 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1188/2379] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:13:13 2022] PUT /identity/v3/projects/e76d5af3829b4fd5bc75fa2594e1ad6f/users/f3abd0b11a9f4e11b7bf95b536c90198/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:13:13.199236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56eafb-2ffe-4169-a01b-37224a9ac8af None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:13.199744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56eafb-2ffe-4169-a01b-37224a9ac8af None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:13.199854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56eafb-2ffe-4169-a01b-37224a9ac8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:13.230493 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-af56eafb-2ffe-4169-a01b-37224a9ac8af None None] MFA Rules not processed for user `f3abd0b11a9f4e11b7bf95b536c90198`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:13:13.231760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-af56eafb-2ffe-4169-a01b-37224a9ac8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:13.276205 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:13:13.303344 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1192/2380] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:13:13 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:13:13.337500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f13ca-fe15-4828-a559-06c4c867ede2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:13.339211 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-589f13ca-fe15-4828-a559-06c4c867ede2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:13.345160 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:13:13.400804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f13ca-fe15-4828-a559-06c4c867ede2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:13.404302 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b9d7e-3c90-47c1-b1d5-39472b357ac5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:13.404900 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b9d7e-3c90-47c1-b1d5-39472b357ac5 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:13.405179 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b9d7e-3c90-47c1-b1d5-39472b357ac5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:13.405453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b9d7e-3c90-47c1-b1d5-39472b357ac5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:13.413951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b9d7e-3c90-47c1-b1d5-39472b357ac5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:13.415367 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b9d7e-3c90-47c1-b1d5-39472b357ac5 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:13.424235 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1189/2381] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:13:13 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:13:13.440188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56eafb-2ffe-4169-a01b-37224a9ac8af None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:13.502586 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56eafb-2ffe-4169-a01b-37224a9ac8af None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:13.507101 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d1d590-aaba-4514-8415-a8741a41b382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:13.508018 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1d590-aaba-4514-8415-a8741a41b382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:13.508302 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1d590-aaba-4514-8415-a8741a41b382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:13.508578 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1d590-aaba-4514-8415-a8741a41b382 admin admin] PATH_INFO: `/v3/users/f3abd0b11a9f4e11b7bf95b536c90198` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:13.509712 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d1d590-aaba-4514-8415-a8741a41b382 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3abd0b11a9f4e11b7bf95b536c90198)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:13.511683 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d1d590-aaba-4514-8415-a8741a41b382 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:13.557340 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-97d1d590-aaba-4514-8415-a8741a41b382 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3abd0b11a9f4e11b7bf95b536c90198'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:13.557985 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-97d1d590-aaba-4514-8415-a8741a41b382 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3abd0b11a9f4e11b7bf95b536c90198'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:13.566610 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-97d1d590-aaba-4514-8415-a8741a41b382 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3abd0b11a9f4e11b7bf95b536c90198'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:13.573553 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-97d1d590-aaba-4514-8415-a8741a41b382 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3abd0b11a9f4e11b7bf95b536c90198'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:13.574204 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-97d1d590-aaba-4514-8415-a8741a41b382 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3abd0b11a9f4e11b7bf95b536c90198'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:13.591834 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1193/2382] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:13:13 2022] DELETE /identity/v3/users/f3abd0b11a9f4e11b7bf95b536c90198 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:13:13.596762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85d311-fa5c-404a-a950-92f7e67c2895 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:13.597081 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85d311-fa5c-404a-a950-92f7e67c2895 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:13.597365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85d311-fa5c-404a-a950-92f7e67c2895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:13.617902 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-fe85d311-fa5c-404a-a950-92f7e67c2895 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:13:13.619111 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fe85d311-fa5c-404a-a950-92f7e67c2895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:13.677824 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1190/2383] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:13:13.687402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d1d590-aaba-4514-8415-a8741a41b382 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:13.688955 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-97d1d590-aaba-4514-8415-a8741a41b382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:13.746118 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d1d590-aaba-4514-8415-a8741a41b382 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:13.749747 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f087cb-9098-490a-ad0d-38de3a7c85f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:13.750386 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f087cb-9098-490a-ad0d-38de3a7c85f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:13.750677 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f087cb-9098-490a-ad0d-38de3a7c85f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:13.751107 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f087cb-9098-490a-ad0d-38de3a7c85f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:13.760329 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f087cb-9098-490a-ad0d-38de3a7c85f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:13.761771 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f087cb-9098-490a-ad0d-38de3a7c85f7 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:13.771756 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1194/2384] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:13:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:13:14.040228 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85d311-fa5c-404a-a950-92f7e67c2895 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:14.041636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fe85d311-fa5c-404a-a950-92f7e67c2895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:14.054374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85d311-fa5c-404a-a950-92f7e67c2895 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:14.059266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078628b-f4ac-4b4d-9786-e0698f3ea420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:14.059980 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078628b-f4ac-4b4d-9786-e0698f3ea420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:14.060572 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078628b-f4ac-4b4d-9786-e0698f3ea420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:14.060572 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078628b-f4ac-4b4d-9786-e0698f3ea420 admin admin] PATH_INFO: `/v3/projects/e76d5af3829b4fd5bc75fa2594e1ad6f` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:14.061629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078628b-f4ac-4b4d-9786-e0698f3ea420 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e76d5af3829b4fd5bc75fa2594e1ad6f)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:14.064300 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078628b-f4ac-4b4d-9786-e0698f3ea420 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:14.109772 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5078628b-f4ac-4b4d-9786-e0698f3ea420 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e76d5af3829b4fd5bc75fa2594e1ad6f', 'request_id': 'req-5078628b-f4ac-4b4d-9786-e0698f3ea420'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:14.117063 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5078628b-f4ac-4b4d-9786-e0698f3ea420 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e76d5af3829b4fd5bc75fa2594e1ad6f', 'request_id': 'req-5078628b-f4ac-4b4d-9786-e0698f3ea420'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:14.122763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5078628b-f4ac-4b4d-9786-e0698f3ea420 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e76d5af3829b4fd5bc75fa2594e1ad6f', 'request_id': 'req-5078628b-f4ac-4b4d-9786-e0698f3ea420'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:14.134589 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5078628b-f4ac-4b4d-9786-e0698f3ea420 admin admin] The token cache is being invalidate because project e76d5af3829b4fd5bc75fa2594e1ad6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:13:14.134876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5078628b-f4ac-4b4d-9786-e0698f3ea420 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:14.135500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-5078628b-f4ac-4b4d-9786-e0698f3ea420 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:14.137039 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1191/2385] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:13:14 2022] DELETE /identity/v3/projects/e76d5af3829b4fd5bc75fa2594e1ad6f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:13:14.142674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa82dd-2604-4b38-b66a-6065d9952b99 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:14.142964 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa82dd-2604-4b38-b66a-6065d9952b99 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:14.143243 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa82dd-2604-4b38-b66a-6065d9952b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:14.151845 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:13:14.164606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-8efa82dd-2604-4b38-b66a-6065d9952b99 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:13:14.165821 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-8efa82dd-2604-4b38-b66a-6065d9952b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:14.224771 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1195/2386] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:13:14.232906 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93f825-1317-48e1-880c-8545dbec4d0b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:14.233195 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93f825-1317-48e1-880c-8545dbec4d0b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:14.233474 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93f825-1317-48e1-880c-8545dbec4d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:14.253663 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ee93f825-1317-48e1-880c-8545dbec4d0b None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:13:14.254857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ee93f825-1317-48e1-880c-8545dbec4d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:14.262887 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1192/2387] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:13:14.267640 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa82dd-2604-4b38-b66a-6065d9952b99 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:14.277411 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa82dd-2604-4b38-b66a-6065d9952b99 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:14.282624 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461c753-a138-437a-9288-a13f1f4ec107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:14.283223 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461c753-a138-437a-9288-a13f1f4ec107 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:14.283509 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461c753-a138-437a-9288-a13f1f4ec107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:14.283818 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461c753-a138-437a-9288-a13f1f4ec107 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:14.284466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461c753-a138-437a-9288-a13f1f4ec107 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:13:14.284773 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461c753-a138-437a-9288-a13f1f4ec107 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:14.286413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461c753-a138-437a-9288-a13f1f4ec107 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:14.298090 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1196/2388] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:13:14.303683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93f825-1317-48e1-880c-8545dbec4d0b None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:14.312952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93f825-1317-48e1-880c-8545dbec4d0b None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:14.317639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a327cb-17ff-488c-a00d-2123208d647a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:14.318240 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a327cb-17ff-488c-a00d-2123208d647a admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:14.318520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a327cb-17ff-488c-a00d-2123208d647a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:14.318792 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a327cb-17ff-488c-a00d-2123208d647a admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:14.319423 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a327cb-17ff-488c-a00d-2123208d647a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:14.321407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a327cb-17ff-488c-a00d-2123208d647a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:14.342953 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:13:14.355732 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1193/2389] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:13:14 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:13:14.357447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461c753-a138-437a-9288-a13f1f4ec107 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:14.370330 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461c753-a138-437a-9288-a13f1f4ec107 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:14.427006 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25136550-9ff8-4263-a789-754a2710f42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:14.427625 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25136550-9ff8-4263-a789-754a2710f42e admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:14.427910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25136550-9ff8-4263-a789-754a2710f42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:14.428182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25136550-9ff8-4263-a789-754a2710f42e admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:14.428801 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25136550-9ff8-4263-a789-754a2710f42e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:14.430736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25136550-9ff8-4263-a789-754a2710f42e admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:14.451363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1197/2390] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:13:14 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:13:14.456271 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a327cb-17ff-488c-a00d-2123208d647a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:14.466620 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a327cb-17ff-488c-a00d-2123208d647a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:14.471950 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5e097-229c-4674-b870-c8ecc8509f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:14.472556 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5e097-229c-4674-b870-c8ecc8509f4d admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:14.472839 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5e097-229c-4674-b870-c8ecc8509f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:14.473111 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5e097-229c-4674-b870-c8ecc8509f4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:14.473757 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5e097-229c-4674-b870-c8ecc8509f4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:14.475415 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5e097-229c-4674-b870-c8ecc8509f4d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:14.486199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1194/2391] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:13:14.490196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25136550-9ff8-4263-a789-754a2710f42e admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:14.500138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25136550-9ff8-4263-a789-754a2710f42e admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:14.505602 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031088d3-9d7d-4ada-a2ed-f99057fb6ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:14.506283 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031088d3-9d7d-4ada-a2ed-f99057fb6ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:14.506639 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031088d3-9d7d-4ada-a2ed-f99057fb6ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:14.506968 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031088d3-9d7d-4ada-a2ed-f99057fb6ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:14.507908 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031088d3-9d7d-4ada-a2ed-f99057fb6ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:14.509492 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031088d3-9d7d-4ada-a2ed-f99057fb6ce8 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:14.521055 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1198/2392] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:13:14.525108 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5e097-229c-4674-b870-c8ecc8509f4d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:14.534928 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5e097-229c-4674-b870-c8ecc8509f4d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:14.539542 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ad3d2-def5-4ffb-b8ba-798d23df627a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:14.540216 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ad3d2-def5-4ffb-b8ba-798d23df627a admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:14.540497 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ad3d2-def5-4ffb-b8ba-798d23df627a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:14.540772 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ad3d2-def5-4ffb-b8ba-798d23df627a admin admin] PATH_INFO: `/v3/projects/efb6f83e694a434c96ea382c8e699e34/users/ba0a42f32da3442a8b5cd0ae8205700f/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:14.555464 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ad3d2-def5-4ffb-b8ba-798d23df627a admin admin] RBAC: Authorizing `identity:create_grant(project_id=efb6f83e694a434c96ea382c8e699e34, user_id=ba0a42f32da3442a8b5cd0ae8205700f, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:14.558186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ad3d2-def5-4ffb-b8ba-798d23df627a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:14.571154 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1195/2393] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:13:14 2022] PUT /identity/v3/projects/efb6f83e694a434c96ea382c8e699e34/users/ba0a42f32da3442a8b5cd0ae8205700f/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:13:14.581063 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb24b24-bdea-4bcd-a929-f1d104937e9d None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:14.581349 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb24b24-bdea-4bcd-a929-f1d104937e9d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:14.581623 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb24b24-bdea-4bcd-a929-f1d104937e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:14.609383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-7eb24b24-bdea-4bcd-a929-f1d104937e9d None None] MFA Rules not processed for user `ba0a42f32da3442a8b5cd0ae8205700f`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:13:14.611375 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7eb24b24-bdea-4bcd-a929-f1d104937e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:14.671966 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:13:14.677007 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1199/2394] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:13:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:13:14.687655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ad3d2-def5-4ffb-b8ba-798d23df627a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:14.689640 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4c0ad3d2-def5-4ffb-b8ba-798d23df627a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:14.746394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ad3d2-def5-4ffb-b8ba-798d23df627a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:14.749961 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198337b4-1350-4013-bc98-a49ca99efd6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:14.750604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198337b4-1350-4013-bc98-a49ca99efd6f service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:14.750879 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198337b4-1350-4013-bc98-a49ca99efd6f service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:14.751188 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198337b4-1350-4013-bc98-a49ca99efd6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:14.759663 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198337b4-1350-4013-bc98-a49ca99efd6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:14.761091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198337b4-1350-4013-bc98-a49ca99efd6f service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:14.769700 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1196/2395] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:13:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:13:17.932207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb24b24-bdea-4bcd-a929-f1d104937e9d None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:17.994966 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb24b24-bdea-4bcd-a929-f1d104937e9d None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:17.999478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca456a-ad19-47b6-9701-4cd0beedeebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:18.000629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca456a-ad19-47b6-9701-4cd0beedeebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:18.000964 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca456a-ad19-47b6-9701-4cd0beedeebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:18.001242 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca456a-ad19-47b6-9701-4cd0beedeebe admin admin] PATH_INFO: `/v3/users/ba0a42f32da3442a8b5cd0ae8205700f` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:18.002373 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca456a-ad19-47b6-9701-4cd0beedeebe admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba0a42f32da3442a8b5cd0ae8205700f)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:18.004343 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca456a-ad19-47b6-9701-4cd0beedeebe admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:18.050856 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-97ca456a-ad19-47b6-9701-4cd0beedeebe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba0a42f32da3442a8b5cd0ae8205700f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:18.051501 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-97ca456a-ad19-47b6-9701-4cd0beedeebe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba0a42f32da3442a8b5cd0ae8205700f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:18.058474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-97ca456a-ad19-47b6-9701-4cd0beedeebe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba0a42f32da3442a8b5cd0ae8205700f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:18.066351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-97ca456a-ad19-47b6-9701-4cd0beedeebe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba0a42f32da3442a8b5cd0ae8205700f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:18.067001 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-97ca456a-ad19-47b6-9701-4cd0beedeebe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba0a42f32da3442a8b5cd0ae8205700f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:18.084022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1200/2396] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:13:17 2022] DELETE /identity/v3/users/ba0a42f32da3442a8b5cd0ae8205700f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:13:18.088502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fd9f5-ddc3-4963-8922-51d0b361f45a None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:18.089022 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fd9f5-ddc3-4963-8922-51d0b361f45a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:18.089130 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fd9f5-ddc3-4963-8922-51d0b361f45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:18.107887 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e03fd9f5-ddc3-4963-8922-51d0b361f45a None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:13:18.109085 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e03fd9f5-ddc3-4963-8922-51d0b361f45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:18.167846 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1197/2397] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:13:18.178166 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca456a-ad19-47b6-9701-4cd0beedeebe admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:18.179628 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-97ca456a-ad19-47b6-9701-4cd0beedeebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:18.237805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca456a-ad19-47b6-9701-4cd0beedeebe admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:18.246865 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214876d-ad4c-4fb2-bf96-8d64d5fb3f6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:18.247497 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214876d-ad4c-4fb2-bf96-8d64d5fb3f6a service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:18.247810 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214876d-ad4c-4fb2-bf96-8d64d5fb3f6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:18.248086 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214876d-ad4c-4fb2-bf96-8d64d5fb3f6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:18.258648 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214876d-ad4c-4fb2-bf96-8d64d5fb3f6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:18.260134 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214876d-ad4c-4fb2-bf96-8d64d5fb3f6a service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:18.269676 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1201/2398] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:13:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:13:18.534938 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fd9f5-ddc3-4963-8922-51d0b361f45a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:18.536436 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e03fd9f5-ddc3-4963-8922-51d0b361f45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:18.552688 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fd9f5-ddc3-4963-8922-51d0b361f45a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:18.557564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b521e-9e53-4261-aee3-206307f9c4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:18.558523 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b521e-9e53-4261-aee3-206307f9c4a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:18.558814 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b521e-9e53-4261-aee3-206307f9c4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:18.559090 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b521e-9e53-4261-aee3-206307f9c4a2 admin admin] PATH_INFO: `/v3/projects/efb6f83e694a434c96ea382c8e699e34` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:18.560310 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b521e-9e53-4261-aee3-206307f9c4a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efb6f83e694a434c96ea382c8e699e34)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:18.562208 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b521e-9e53-4261-aee3-206307f9c4a2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:18.608311 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-138b521e-9e53-4261-aee3-206307f9c4a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efb6f83e694a434c96ea382c8e699e34', 'request_id': 'req-138b521e-9e53-4261-aee3-206307f9c4a2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:18.615469 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-138b521e-9e53-4261-aee3-206307f9c4a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efb6f83e694a434c96ea382c8e699e34', 'request_id': 'req-138b521e-9e53-4261-aee3-206307f9c4a2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:18.620988 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:13:18.622258 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-138b521e-9e53-4261-aee3-206307f9c4a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efb6f83e694a434c96ea382c8e699e34', 'request_id': 'req-138b521e-9e53-4261-aee3-206307f9c4a2'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:18.637000 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-138b521e-9e53-4261-aee3-206307f9c4a2 admin admin] The token cache is being invalidate because project efb6f83e694a434c96ea382c8e699e34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:13:18.637609 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-138b521e-9e53-4261-aee3-206307f9c4a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:18.638415 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-138b521e-9e53-4261-aee3-206307f9c4a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:13:18.639985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1198/2399] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:13:18 2022] DELETE /identity/v3/projects/efb6f83e694a434c96ea382c8e699e34 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:13:18.644835 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b7107-152a-42af-96a6-918b2f944206 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:18.645150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b7107-152a-42af-96a6-918b2f944206 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:18.645427 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b7107-152a-42af-96a6-918b2f944206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:18.665312 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-204b7107-152a-42af-96a6-918b2f944206 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:13:18.666506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-204b7107-152a-42af-96a6-918b2f944206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:18.683779 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:13:18.728220 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1202/2400] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:13:18.736824 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a53092-c782-44e8-b34f-a8fa04fe910e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:18.737124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a53092-c782-44e8-b34f-a8fa04fe910e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:18.737388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a53092-c782-44e8-b34f-a8fa04fe910e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:18.756737 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-24a53092-c782-44e8-b34f-a8fa04fe910e None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:13:18.758185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-24a53092-c782-44e8-b34f-a8fa04fe910e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:18.766473 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1199/2401] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:13:18.770536 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b7107-152a-42af-96a6-918b2f944206 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:18.779886 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b7107-152a-42af-96a6-918b2f944206 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:18.785371 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba7bb3-473c-4a45-9e19-aa9a426a9d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:18.785958 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba7bb3-473c-4a45-9e19-aa9a426a9d82 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:18.786239 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba7bb3-473c-4a45-9e19-aa9a426a9d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:18.786513 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba7bb3-473c-4a45-9e19-aa9a426a9d82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:18.787152 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba7bb3-473c-4a45-9e19-aa9a426a9d82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:13:18.787452 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba7bb3-473c-4a45-9e19-aa9a426a9d82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:18.789072 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba7bb3-473c-4a45-9e19-aa9a426a9d82 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:18.800477 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1203/2402] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:13:18.804649 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a53092-c782-44e8-b34f-a8fa04fe910e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:18.813711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a53092-c782-44e8-b34f-a8fa04fe910e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:18.818394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f00d17e-d6e8-4c44-bf99-f8adcd7ed046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:18.819012 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00d17e-d6e8-4c44-bf99-f8adcd7ed046 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:18.819287 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00d17e-d6e8-4c44-bf99-f8adcd7ed046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:18.819576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00d17e-d6e8-4c44-bf99-f8adcd7ed046 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:18.820317 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f00d17e-d6e8-4c44-bf99-f8adcd7ed046 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:18.822200 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f00d17e-d6e8-4c44-bf99-f8adcd7ed046 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:18.849881 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1200/2403] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:13:18 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:13:18.854127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba7bb3-473c-4a45-9e19-aa9a426a9d82 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:18.864201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba7bb3-473c-4a45-9e19-aa9a426a9d82 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:18.920343 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487c532-961d-4084-8beb-b81112a260f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:18.920947 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487c532-961d-4084-8beb-b81112a260f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:18.921229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487c532-961d-4084-8beb-b81112a260f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:18.921505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487c532-961d-4084-8beb-b81112a260f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:18.922218 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487c532-961d-4084-8beb-b81112a260f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:18.924318 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487c532-961d-4084-8beb-b81112a260f3 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:18.945038 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1204/2404] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:13:18 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:13:18.949234 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f00d17e-d6e8-4c44-bf99-f8adcd7ed046 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:18.960657 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f00d17e-d6e8-4c44-bf99-f8adcd7ed046 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:18.965386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc35a61-2e89-4908-882c-93dbcb6632ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:18.965987 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc35a61-2e89-4908-882c-93dbcb6632ea admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:18.966269 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc35a61-2e89-4908-882c-93dbcb6632ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:18.966544 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc35a61-2e89-4908-882c-93dbcb6632ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:18.967191 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc35a61-2e89-4908-882c-93dbcb6632ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:18.968839 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc35a61-2e89-4908-882c-93dbcb6632ea admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:18.979326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1201/2405] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:13:18.983373 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487c532-961d-4084-8beb-b81112a260f3 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:18.998183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487c532-961d-4084-8beb-b81112a260f3 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:19.003340 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1a73b-bc5b-459b-80fe-e5af9ea030c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:19.003989 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1a73b-bc5b-459b-80fe-e5af9ea030c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:19.004271 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1a73b-bc5b-459b-80fe-e5af9ea030c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:19.004548 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1a73b-bc5b-459b-80fe-e5af9ea030c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:19.005189 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b1a73b-bc5b-459b-80fe-e5af9ea030c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:19.006801 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b1a73b-bc5b-459b-80fe-e5af9ea030c5 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:19.019448 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1205/2406] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:13:19.023817 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc35a61-2e89-4908-882c-93dbcb6632ea admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:19.034581 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc35a61-2e89-4908-882c-93dbcb6632ea admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:19.039141 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21545db9-ae1a-4f0d-95dc-5dfc8382f0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:19.039816 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21545db9-ae1a-4f0d-95dc-5dfc8382f0a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:19.040095 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21545db9-ae1a-4f0d-95dc-5dfc8382f0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:19.040364 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21545db9-ae1a-4f0d-95dc-5dfc8382f0a7 admin admin] PATH_INFO: `/v3/projects/f93905695d7a4f8495534c38a7bebfa4/users/f8d750f711d34f80816f124cb3d1f4b4/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:19.056066 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21545db9-ae1a-4f0d-95dc-5dfc8382f0a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f93905695d7a4f8495534c38a7bebfa4, user_id=f8d750f711d34f80816f124cb3d1f4b4, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:19.058788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21545db9-ae1a-4f0d-95dc-5dfc8382f0a7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:19.072091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1202/2407] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:13:19 2022] PUT /identity/v3/projects/f93905695d7a4f8495534c38a7bebfa4/users/f8d750f711d34f80816f124cb3d1f4b4/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:13:19.077231 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a18b1d-8e8a-4faa-81a9-6b17fdd79ef0 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:19.077522 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a18b1d-8e8a-4faa-81a9-6b17fdd79ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:19.077801 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a18b1d-8e8a-4faa-81a9-6b17fdd79ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:19.097603 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-72a18b1d-8e8a-4faa-81a9-6b17fdd79ef0 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:13:19.098801 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-72a18b1d-8e8a-4faa-81a9-6b17fdd79ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:19.157640 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1206/2408] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:13:19.167477 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21545db9-ae1a-4f0d-95dc-5dfc8382f0a7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:19.169196 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-21545db9-ae1a-4f0d-95dc-5dfc8382f0a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:19.226019 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21545db9-ae1a-4f0d-95dc-5dfc8382f0a7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:19.229556 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00287dd-2b7f-4bf8-bb32-a826434dab02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:19.230159 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00287dd-2b7f-4bf8-bb32-a826434dab02 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:19.230438 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00287dd-2b7f-4bf8-bb32-a826434dab02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:19.230712 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00287dd-2b7f-4bf8-bb32-a826434dab02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:19.240360 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00287dd-2b7f-4bf8-bb32-a826434dab02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:19.241810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00287dd-2b7f-4bf8-bb32-a826434dab02 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:19.251942 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1203/2409] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:13:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:13:22.964797 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a0e51-05c3-4093-9c87-79ae9a223977 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:22.965324 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a0e51-05c3-4093-9c87-79ae9a223977 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:22.965324 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a0e51-05c3-4093-9c87-79ae9a223977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:22.994982 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-5e5a0e51-05c3-4093-9c87-79ae9a223977 None None] MFA Rules not processed for user `f8d750f711d34f80816f124cb3d1f4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:13:22.996254 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5e5a0e51-05c3-4093-9c87-79ae9a223977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:23.013865 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:13:23.066471 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1207/2410] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:13:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:13:23.071315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00287dd-2b7f-4bf8-bb32-a826434dab02 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:23.082477 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00287dd-2b7f-4bf8-bb32-a826434dab02 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:23.088233 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a6e38-1070-4979-9751-6ec753b81ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:23.088863 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a6e38-1070-4979-9751-6ec753b81ad7 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:23.089144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a6e38-1070-4979-9751-6ec753b81ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:23.089417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a6e38-1070-4979-9751-6ec753b81ad7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:23.095779 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a6e38-1070-4979-9751-6ec753b81ad7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:23.095779 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a6e38-1070-4979-9751-6ec753b81ad7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:23.122568 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1204/2411] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:13:23 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:13:23.123814 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a0e51-05c3-4093-9c87-79ae9a223977 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:23.136112 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a0e51-05c3-4093-9c87-79ae9a223977 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:23.213285 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef3a9f-cf46-425c-9a18-75ec4e030ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:23.213939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef3a9f-cf46-425c-9a18-75ec4e030ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:23.214218 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef3a9f-cf46-425c-9a18-75ec4e030ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:23.214488 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef3a9f-cf46-425c-9a18-75ec4e030ca9 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:23.215127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef3a9f-cf46-425c-9a18-75ec4e030ca9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:23.217158 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef3a9f-cf46-425c-9a18-75ec4e030ca9 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:23.247884 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a6e38-1070-4979-9751-6ec753b81ad7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:23.248217 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1208/2412] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:13:23 2022] POST /identity/v3/users => generated 299 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:13:23.260998 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a6e38-1070-4979-9751-6ec753b81ad7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:23.267273 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9985011-3949-433c-b3e8-f6c1decfc7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:23.267953 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9985011-3949-433c-b3e8-f6c1decfc7c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:23.268240 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9985011-3949-433c-b3e8-f6c1decfc7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:23.272923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9985011-3949-433c-b3e8-f6c1decfc7c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:23.273789 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9985011-3949-433c-b3e8-f6c1decfc7c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:23.275503 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9985011-3949-433c-b3e8-f6c1decfc7c6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:23.293861 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1205/2413] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:13:23.304646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef3a9f-cf46-425c-9a18-75ec4e030ca9 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:23.319013 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef3a9f-cf46-425c-9a18-75ec4e030ca9 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:23.324196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a9fb0-979e-4365-86f2-19fef0b4d868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:23.324945 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a9fb0-979e-4365-86f2-19fef0b4d868 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:23.325315 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a9fb0-979e-4365-86f2-19fef0b4d868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:23.325652 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a9fb0-979e-4365-86f2-19fef0b4d868 admin admin] PATH_INFO: `/v3/projects/cae52d32e65b4630be939de99ecc9d0f/users/fb0bb8980df24136942f13f8904b3e2a/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:23.349099 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a9fb0-979e-4365-86f2-19fef0b4d868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cae52d32e65b4630be939de99ecc9d0f, user_id=fb0bb8980df24136942f13f8904b3e2a, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:23.351871 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a9fb0-979e-4365-86f2-19fef0b4d868 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:23.369048 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1209/2414] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:13:23 2022] PUT /identity/v3/projects/cae52d32e65b4630be939de99ecc9d0f/users/fb0bb8980df24136942f13f8904b3e2a/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:13:23.372910 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9985011-3949-433c-b3e8-f6c1decfc7c6 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:23.385225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9985011-3949-433c-b3e8-f6c1decfc7c6 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:23.444212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:13:23.451111 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05e7d1-81ca-4968-8859-09048af08e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:23.451876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05e7d1-81ca-4968-8859-09048af08e3e admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:23.452336 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05e7d1-81ca-4968-8859-09048af08e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:23.452686 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05e7d1-81ca-4968-8859-09048af08e3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:23.453472 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05e7d1-81ca-4968-8859-09048af08e3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:23.455453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05e7d1-81ca-4968-8859-09048af08e3e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:23.465987 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1206/2415] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:13:23.470668 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a9fb0-979e-4365-86f2-19fef0b4d868 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:23.481544 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a9fb0-979e-4365-86f2-19fef0b4d868 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:23.487174 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a546f0-dcfd-45a3-96e0-3ccb7bdab4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:23.487895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a546f0-dcfd-45a3-96e0-3ccb7bdab4f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:23.488182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a546f0-dcfd-45a3-96e0-3ccb7bdab4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:23.488461 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a546f0-dcfd-45a3-96e0-3ccb7bdab4f2 admin admin] PATH_INFO: `/v3/projects/cae52d32e65b4630be939de99ecc9d0f/users/fb0bb8980df24136942f13f8904b3e2a/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:23.490424 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a546f0-dcfd-45a3-96e0-3ccb7bdab4f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cae52d32e65b4630be939de99ecc9d0f, user_id=fb0bb8980df24136942f13f8904b3e2a, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:23.493171 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a546f0-dcfd-45a3-96e0-3ccb7bdab4f2 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:23.505180 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1210/2416] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:13:23 2022] PUT /identity/v3/projects/cae52d32e65b4630be939de99ecc9d0f/users/fb0bb8980df24136942f13f8904b3e2a/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:13:27.929711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08434917-9f85-462f-9fbc-10e02aa74e88 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:27.930245 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08434917-9f85-462f-9fbc-10e02aa74e88 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:27.930345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08434917-9f85-462f-9fbc-10e02aa74e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:27.960720 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-08434917-9f85-462f-9fbc-10e02aa74e88 None None] MFA Rules not processed for user `fb0bb8980df24136942f13f8904b3e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:13:27.961903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-08434917-9f85-462f-9fbc-10e02aa74e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:28.038016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1207/2417] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:13:27 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:13:28.050529 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a546f0-dcfd-45a3-96e0-3ccb7bdab4f2 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:28.052826 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-49a546f0-dcfd-45a3-96e0-3ccb7bdab4f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:28.117660 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a546f0-dcfd-45a3-96e0-3ccb7bdab4f2 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:28.121636 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8f0e7-5c70-404e-9a92-4d65adb4d418 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89ff357209194276bc079d0c05898747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:28.122260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8f0e7-5c70-404e-9a92-4d65adb4d418 service glance] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:28.122541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8f0e7-5c70-404e-9a92-4d65adb4d418 service glance] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:28.122815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8f0e7-5c70-404e-9a92-4d65adb4d418 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:28.150241 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:13:28.186464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8f0e7-5c70-404e-9a92-4d65adb4d418 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:28.188694 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8f0e7-5c70-404e-9a92-4d65adb4d418 service glance] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:28.198590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1211/2418] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:13:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:13:28.356672 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b04b5-a8fc-4fda-9cdc-f23cef611f02 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:28.356902 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b04b5-a8fc-4fda-9cdc-f23cef611f02 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:28.357182 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b04b5-a8fc-4fda-9cdc-f23cef611f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:28.395913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a42b04b5-a8fc-4fda-9cdc-f23cef611f02 None None] MFA Rules not processed for user `ceccc909529042539dd2d5015058d8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:13:28.397128 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a42b04b5-a8fc-4fda-9cdc-f23cef611f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:28.465678 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1208/2419] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 01:13:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:13:28.519609 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8f0e7-5c70-404e-9a92-4d65adb4d418 service glance] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:28.521107 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-20f8f0e7-5c70-404e-9a92-4d65adb4d418 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:28.595801 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8f0e7-5c70-404e-9a92-4d65adb4d418 service glance] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:28.601037 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4333158e-29d2-49a0-83b3-27c65738be89 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e52776d9b4894af9b85a24136a8255e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:28.601663 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333158e-29d2-49a0-83b3-27c65738be89 service swift] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:28.601942 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333158e-29d2-49a0-83b3-27c65738be89 service swift] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:28.602215 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333158e-29d2-49a0-83b3-27c65738be89 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:28.611644 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4333158e-29d2-49a0-83b3-27c65738be89 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:28.613388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4333158e-29d2-49a0-83b3-27c65738be89 service swift] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:28.622427 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1212/2420] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 01:13:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 01:13:37.478086 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2fab3-5b3e-46ef-bba1-24a5a3b1ff3a None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:37.478670 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2fab3-5b3e-46ef-bba1-24a5a3b1ff3a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:37.478670 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2fab3-5b3e-46ef-bba1-24a5a3b1ff3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:37.499586 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a2f2fab3-5b3e-46ef-bba1-24a5a3b1ff3a None None] MFA Rules not processed for user `f8d750f711d34f80816f124cb3d1f4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:13:37.501164 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a2f2fab3-5b3e-46ef-bba1-24a5a3b1ff3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:37.508333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1209/2421] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:13:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:13:38.618545 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=48 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:13:39.696396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb26b4b-04e3-448a-b473-c6838c1af5fd None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:39.696996 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb26b4b-04e3-448a-b473-c6838c1af5fd None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:39.697118 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb26b4b-04e3-448a-b473-c6838c1af5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:39.716642 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-0eb26b4b-04e3-448a-b473-c6838c1af5fd None None] MFA Rules not processed for user `ceccc909529042539dd2d5015058d8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:13:39.717875 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0eb26b4b-04e3-448a-b473-c6838c1af5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:13:39.725795 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1213/2422] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 01:13:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:13:39.739171 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2fab3-5b3e-46ef-bba1-24a5a3b1ff3a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:13:39.751663 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2fab3-5b3e-46ef-bba1-24a5a3b1ff3a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:13:39.756635 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff9d26-1cfe-4b39-a85a-fb12d803e1a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e52776d9b4894af9b85a24136a8255e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:13:39.757527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff9d26-1cfe-4b39-a85a-fb12d803e1a4 service swift] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:13:39.757804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff9d26-1cfe-4b39-a85a-fb12d803e1a4 service swift] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:13:39.758075 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff9d26-1cfe-4b39-a85a-fb12d803e1a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:13:39.766926 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff9d26-1cfe-4b39-a85a-fb12d803e1a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:13:39.768643 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff9d26-1cfe-4b39-a85a-fb12d803e1a4 service swift] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:13:39.776632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1210/2423] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 01:13:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 01:13:49.710851 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:13:49.773401 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=78 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:14:06.778117 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b7802-6d37-44bf-84b0-4b1012dcd4c2 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:14:06.778690 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b7802-6d37-44bf-84b0-4b1012dcd4c2 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:14:06.778982 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b7802-6d37-44bf-84b0-4b1012dcd4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:14:06.799154 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-408b7802-6d37-44bf-84b0-4b1012dcd4c2 None None] MFA Rules not processed for user `51768890f19140b6819ef89f5bf610c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:14:06.800590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-408b7802-6d37-44bf-84b0-4b1012dcd4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:14:06.870220 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1214/2424] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:14:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:14:14.557351 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff9d26-1cfe-4b39-a85a-fb12d803e1a4 service swift] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:14:14.613509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff9d26-1cfe-4b39-a85a-fb12d803e1a4 service swift] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:14:14.618104 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff3e38-e472-4da9-b782-611a7098c436 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:14:14.618747 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff3e38-e472-4da9-b782-611a7098c436 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:14:14.619017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff3e38-e472-4da9-b782-611a7098c436 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:14:14.619283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff3e38-e472-4da9-b782-611a7098c436 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:14:14.628454 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff3e38-e472-4da9-b782-611a7098c436 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:14:14.629886 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff3e38-e472-4da9-b782-611a7098c436 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:14:14.640358 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1211/2425] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:14:14 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:14:16.856077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=25 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:14:20.554357 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80430f1-86dd-4588-a3a9-ea56556c7888 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:14:20.554947 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80430f1-86dd-4588-a3a9-ea56556c7888 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:14:20.554947 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80430f1-86dd-4588-a3a9-ea56556c7888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:14:20.573804 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-b80430f1-86dd-4588-a3a9-ea56556c7888 None None] MFA Rules not processed for user `ceccc909529042539dd2d5015058d8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:14:20.575272 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b80430f1-86dd-4588-a3a9-ea56556c7888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:14:20.584515 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1215/2426] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 01:14:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:14:20.596995 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff3e38-e472-4da9-b782-611a7098c436 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:14:20.609140 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff3e38-e472-4da9-b782-611a7098c436 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:14:20.614171 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb7c41-f03a-49e7-8fed-67635904caaa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e52776d9b4894af9b85a24136a8255e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:14:20.614776 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb7c41-f03a-49e7-8fed-67635904caaa service swift] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:14:20.615055 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb7c41-f03a-49e7-8fed-67635904caaa service swift] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:14:20.615333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb7c41-f03a-49e7-8fed-67635904caaa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:14:20.624299 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb7c41-f03a-49e7-8fed-67635904caaa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:14:20.625921 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb7c41-f03a-49e7-8fed-67635904caaa service swift] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:14:20.634836 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1212/2427] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 01:14:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 01:14:30.568244 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:14:30.630857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=32 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:14:37.412644 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa7e3d-b4b1-4868-9d8e-cde558fb6b95 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:14:37.413125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa7e3d-b4b1-4868-9d8e-cde558fb6b95 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:14:37.413241 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa7e3d-b4b1-4868-9d8e-cde558fb6b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:14:37.433924 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-fffa7e3d-b4b1-4868-9d8e-cde558fb6b95 None None] MFA Rules not processed for user `ceccc909529042539dd2d5015058d8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:14:37.435140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-fffa7e3d-b4b1-4868-9d8e-cde558fb6b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:14:37.442416 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1216/2428] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 01:14:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:14:37.455171 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb7c41-f03a-49e7-8fed-67635904caaa service swift] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:14:37.466384 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb7c41-f03a-49e7-8fed-67635904caaa service swift] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:14:37.471448 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a278631-da7c-4bf7-a11d-a4eba5b64df6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e52776d9b4894af9b85a24136a8255e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:14:37.472256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a278631-da7c-4bf7-a11d-a4eba5b64df6 service swift] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:14:37.472540 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a278631-da7c-4bf7-a11d-a4eba5b64df6 service swift] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:14:37.472814 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a278631-da7c-4bf7-a11d-a4eba5b64df6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:14:37.481973 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a278631-da7c-4bf7-a11d-a4eba5b64df6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:14:37.483598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a278631-da7c-4bf7-a11d-a4eba5b64df6 service swift] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:14:37.491793 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1213/2429] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 01:14:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 01:14:39.041740 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa7e3d-b4b1-4868-9d8e-cde558fb6b95 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:14:39.051627 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa7e3d-b4b1-4868-9d8e-cde558fb6b95 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:14:39.055725 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c995d6-286f-4938-b390-0e1a97bb2a9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:14:39.056560 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c995d6-286f-4938-b390-0e1a97bb2a9f service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:14:39.057032 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c995d6-286f-4938-b390-0e1a97bb2a9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:14:39.057473 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c995d6-286f-4938-b390-0e1a97bb2a9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:14:39.059428 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-52c995d6-286f-4938-b390-0e1a97bb2a9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:14:39.146828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c995d6-286f-4938-b390-0e1a97bb2a9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:14:39.148665 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c995d6-286f-4938-b390-0e1a97bb2a9f service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:14:39.172248 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1217/2430] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:14:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:14:39.454365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a278631-da7c-4bf7-a11d-a4eba5b64df6 service swift] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:14:39.456430 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3a278631-da7c-4bf7-a11d-a4eba5b64df6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:14:39.522098 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a278631-da7c-4bf7-a11d-a4eba5b64df6 service swift] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:14:39.526654 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da956009-736d-484e-b405-ba3a9dd56dfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:14:39.527287 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da956009-736d-484e-b405-ba3a9dd56dfb service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:14:39.527593 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da956009-736d-484e-b405-ba3a9dd56dfb service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:14:39.527867 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da956009-736d-484e-b405-ba3a9dd56dfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:14:39.529784 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-da956009-736d-484e-b405-ba3a9dd56dfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:14:39.544340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da956009-736d-484e-b405-ba3a9dd56dfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:14:39.545799 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da956009-736d-484e-b405-ba3a9dd56dfb service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:14:39.558987 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1214/2431] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:14:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:14:49.168925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=46 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:14:49.552934 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=32 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:14:54.197817 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c995d6-286f-4938-b390-0e1a97bb2a9f service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:14:54.211969 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c995d6-286f-4938-b390-0e1a97bb2a9f service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:14:54.215703 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b5aa5-66e5-475c-ae1c-61696bc40922 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:14:54.216314 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b5aa5-66e5-475c-ae1c-61696bc40922 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:14:54.216595 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b5aa5-66e5-475c-ae1c-61696bc40922 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:14:54.216868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b5aa5-66e5-475c-ae1c-61696bc40922 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:14:54.218770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-3f6b5aa5-66e5-475c-ae1c-61696bc40922 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:14:54.305086 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b5aa5-66e5-475c-ae1c-61696bc40922 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:14:54.306537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b5aa5-66e5-475c-ae1c-61696bc40922 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:14:54.325203 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1218/2432] 10.222.0.42 () {60 vars in 1396 bytes} [Wed Sep 14 01:14:54 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3254 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:14:59.801698 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da956009-736d-484e-b405-ba3a9dd56dfb service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:14:59.813947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da956009-736d-484e-b405-ba3a9dd56dfb service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:14:59.817575 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994877e1-8395-4cc0-8ef9-85509e91fbe5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89ff357209194276bc079d0c05898747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:14:59.818178 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994877e1-8395-4cc0-8ef9-85509e91fbe5 service glance] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:14:59.818952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994877e1-8395-4cc0-8ef9-85509e91fbe5 service glance] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:14:59.819232 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994877e1-8395-4cc0-8ef9-85509e91fbe5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:14:59.821025 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-994877e1-8395-4cc0-8ef9-85509e91fbe5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:14:59.902639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994877e1-8395-4cc0-8ef9-85509e91fbe5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:14:59.904159 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994877e1-8395-4cc0-8ef9-85509e91fbe5 service glance] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:14:59.922502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1215/2433] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:14:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:14:59.952766 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803f97c-f668-4bd9-8bb4-6f99b2221838 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:14:59.953236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803f97c-f668-4bd9-8bb4-6f99b2221838 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:14:59.953335 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803f97c-f668-4bd9-8bb4-6f99b2221838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:14:59.972840 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d803f97c-f668-4bd9-8bb4-6f99b2221838 None None] MFA Rules not processed for user `ceccc909529042539dd2d5015058d8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:14:59.974030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d803f97c-f668-4bd9-8bb4-6f99b2221838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:14:59.981252 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1219/2434] 10.222.0.42 () {58 vars in 967 bytes} [Wed Sep 14 01:14:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:14:59.993363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994877e1-8395-4cc0-8ef9-85509e91fbe5 service glance] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:00.005399 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994877e1-8395-4cc0-8ef9-85509e91fbe5 service glance] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:00.011228 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29b6e8-0719-4f0d-8254-d0b9e88fba12 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e52776d9b4894af9b85a24136a8255e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:00.011862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29b6e8-0719-4f0d-8254-d0b9e88fba12 service swift] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:00.012147 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29b6e8-0719-4f0d-8254-d0b9e88fba12 service swift] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:00.012423 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29b6e8-0719-4f0d-8254-d0b9e88fba12 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:00.020395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd29b6e8-0719-4f0d-8254-d0b9e88fba12 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:00.022673 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd29b6e8-0719-4f0d-8254-d0b9e88fba12 service swift] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:00.030622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1216/2435] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 01:14:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 01:15:09.967605 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=44 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:15:10.027367 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=47 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:15:24.572576 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803f97c-f668-4bd9-8bb4-6f99b2221838 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:24.582378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803f97c-f668-4bd9-8bb4-6f99b2221838 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:24.586136 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989824f-7578-40df-b16e-923f0a0396d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:24.586782 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989824f-7578-40df-b16e-923f0a0396d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:24.587069 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989824f-7578-40df-b16e-923f0a0396d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:24.587353 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989824f-7578-40df-b16e-923f0a0396d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:24.589422 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1989824f-7578-40df-b16e-923f0a0396d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:24.657116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989824f-7578-40df-b16e-923f0a0396d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:24.658483 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989824f-7578-40df-b16e-923f0a0396d3 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:24.669516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1220/2436] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:15:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:15:27.841261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29b6e8-0719-4f0d-8254-d0b9e88fba12 service swift] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:27.842733 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bd29b6e8-0719-4f0d-8254-d0b9e88fba12 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:27.922556 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989824f-7578-40df-b16e-923f0a0396d3 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:27.924130 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1989824f-7578-40df-b16e-923f0a0396d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:27.940828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989824f-7578-40df-b16e-923f0a0396d3 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:27.945494 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511ed2e-cd4c-45d2-bcbd-caa8008707f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:27.946161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511ed2e-cd4c-45d2-bcbd-caa8008707f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:27.946440 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511ed2e-cd4c-45d2-bcbd-caa8008707f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:27.946717 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511ed2e-cd4c-45d2-bcbd-caa8008707f0 admin admin] PATH_INFO: `/v3/users/2eca12e1e0a64eeeb183a328f338f54d` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:27.947877 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511ed2e-cd4c-45d2-bcbd-caa8008707f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eca12e1e0a64eeeb183a328f338f54d)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:27.948775 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29b6e8-0719-4f0d-8254-d0b9e88fba12 service swift] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:27.949807 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511ed2e-cd4c-45d2-bcbd-caa8008707f0 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:27.953865 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0d848-5cc1-48e9-8b7a-f99f701a0ae6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18798838de284c84b5b838b3530a2c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:27.954476 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0d848-5cc1-48e9-8b7a-f99f701a0ae6 service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:27.954774 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0d848-5cc1-48e9-8b7a-f99f701a0ae6 service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:27.955036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0d848-5cc1-48e9-8b7a-f99f701a0ae6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:27.956806 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f3f0d848-5cc1-48e9-8b7a-f99f701a0ae6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:27.969599 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0d848-5cc1-48e9-8b7a-f99f701a0ae6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:27.971219 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f0d848-5cc1-48e9-8b7a-f99f701a0ae6 service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:27.982627 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1217/2437] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:15:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:15:28.003818 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8511ed2e-cd4c-45d2-bcbd-caa8008707f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eca12e1e0a64eeeb183a328f338f54d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:28.004489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8511ed2e-cd4c-45d2-bcbd-caa8008707f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eca12e1e0a64eeeb183a328f338f54d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:28.012862 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8511ed2e-cd4c-45d2-bcbd-caa8008707f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eca12e1e0a64eeeb183a328f338f54d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:28.020077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8511ed2e-cd4c-45d2-bcbd-caa8008707f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eca12e1e0a64eeeb183a328f338f54d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:28.020734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-8511ed2e-cd4c-45d2-bcbd-caa8008707f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eca12e1e0a64eeeb183a328f338f54d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:28.038159 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1221/2438] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:15:27 2022] DELETE /identity/v3/users/2eca12e1e0a64eeeb183a328f338f54d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:15:28.213043 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0d848-5cc1-48e9-8b7a-f99f701a0ae6 service placement] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:28.224371 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0d848-5cc1-48e9-8b7a-f99f701a0ae6 service placement] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:28.284513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908bcb3-fd4f-4ed6-89a9-f9e8ded270c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:28.285142 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908bcb3-fd4f-4ed6-89a9-f9e8ded270c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:28.285449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908bcb3-fd4f-4ed6-89a9-f9e8ded270c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:28.285720 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908bcb3-fd4f-4ed6-89a9-f9e8ded270c8 admin admin] PATH_INFO: `/v3/projects/25c229f6a4574619a73b2780d2a6528b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:28.286817 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908bcb3-fd4f-4ed6-89a9-f9e8ded270c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25c229f6a4574619a73b2780d2a6528b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:28.288837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908bcb3-fd4f-4ed6-89a9-f9e8ded270c8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:28.335147 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d908bcb3-fd4f-4ed6-89a9-f9e8ded270c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25c229f6a4574619a73b2780d2a6528b', 'request_id': 'req-d908bcb3-fd4f-4ed6-89a9-f9e8ded270c8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:28.341773 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d908bcb3-fd4f-4ed6-89a9-f9e8ded270c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25c229f6a4574619a73b2780d2a6528b', 'request_id': 'req-d908bcb3-fd4f-4ed6-89a9-f9e8ded270c8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:28.347459 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d908bcb3-fd4f-4ed6-89a9-f9e8ded270c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25c229f6a4574619a73b2780d2a6528b', 'request_id': 'req-d908bcb3-fd4f-4ed6-89a9-f9e8ded270c8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:28.359057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d908bcb3-fd4f-4ed6-89a9-f9e8ded270c8 admin admin] The token cache is being invalidate because project 25c229f6a4574619a73b2780d2a6528b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:15:28.359340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d908bcb3-fd4f-4ed6-89a9-f9e8ded270c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:28.359994 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d908bcb3-fd4f-4ed6-89a9-f9e8ded270c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:28.361452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1218/2439] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:15:28 2022] DELETE /identity/v3/projects/25c229f6a4574619a73b2780d2a6528b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:15:28.366598 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249acef4-c422-4c91-a491-05d9f8a55e2b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:28.366896 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249acef4-c422-4c91-a491-05d9f8a55e2b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:28.367203 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249acef4-c422-4c91-a491-05d9f8a55e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:28.387518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-249acef4-c422-4c91-a491-05d9f8a55e2b None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:15:28.388752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-249acef4-c422-4c91-a491-05d9f8a55e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:28.447486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1222/2440] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:15:28.455194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078985b0-7283-4bb1-be31-a168e8ab5dd2 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:28.455487 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078985b0-7283-4bb1-be31-a168e8ab5dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:28.455819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078985b0-7283-4bb1-be31-a168e8ab5dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:28.475486 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-078985b0-7283-4bb1-be31-a168e8ab5dd2 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:15:28.476789 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-078985b0-7283-4bb1-be31-a168e8ab5dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:28.485415 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1219/2441] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:15:28.489616 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249acef4-c422-4c91-a491-05d9f8a55e2b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:28.500118 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249acef4-c422-4c91-a491-05d9f8a55e2b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:28.504688 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f8a25-056c-4777-af7b-46a750a67d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:28.505318 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f8a25-056c-4777-af7b-46a750a67d3e admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:28.505601 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f8a25-056c-4777-af7b-46a750a67d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:28.505875 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f8a25-056c-4777-af7b-46a750a67d3e admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:28.506778 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f8a25-056c-4777-af7b-46a750a67d3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:15:28.507084 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f8a25-056c-4777-af7b-46a750a67d3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:28.508736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f8a25-056c-4777-af7b-46a750a67d3e admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:28.510203 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:15:28.521714 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1223/2442] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:15:28.525920 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078985b0-7283-4bb1-be31-a168e8ab5dd2 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:28.535071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078985b0-7283-4bb1-be31-a168e8ab5dd2 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:28.540313 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd2ff8-b59e-4ae1-afcc-f97623e7b86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:28.541005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd2ff8-b59e-4ae1-afcc-f97623e7b86a admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:28.541293 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd2ff8-b59e-4ae1-afcc-f97623e7b86a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:28.541570 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd2ff8-b59e-4ae1-afcc-f97623e7b86a admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:28.542215 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd2ff8-b59e-4ae1-afcc-f97623e7b86a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:28.544177 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd2ff8-b59e-4ae1-afcc-f97623e7b86a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:28.556304 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:15:28.581632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1220/2443] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:15:28 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:15:28.585771 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f8a25-056c-4777-af7b-46a750a67d3e admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:28.596489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f8a25-056c-4777-af7b-46a750a67d3e admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:28.653445 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9063de04-d740-44bc-965f-d78d90806c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:28.654280 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063de04-d740-44bc-965f-d78d90806c41 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:28.654703 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063de04-d740-44bc-965f-d78d90806c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:28.654984 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063de04-d740-44bc-965f-d78d90806c41 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:28.655938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9063de04-d740-44bc-965f-d78d90806c41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:28.657859 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9063de04-d740-44bc-965f-d78d90806c41 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:28.682421 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1224/2444] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:15:28 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:15:28.683899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd2ff8-b59e-4ae1-afcc-f97623e7b86a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:28.694239 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd2ff8-b59e-4ae1-afcc-f97623e7b86a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:28.699129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03914a32-0b46-420b-9d88-48673ea5f485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:28.699777 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03914a32-0b46-420b-9d88-48673ea5f485 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:28.700060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03914a32-0b46-420b-9d88-48673ea5f485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:28.700341 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03914a32-0b46-420b-9d88-48673ea5f485 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:28.700978 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03914a32-0b46-420b-9d88-48673ea5f485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:28.702581 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03914a32-0b46-420b-9d88-48673ea5f485 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:28.713017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1221/2445] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:15:28.717034 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9063de04-d740-44bc-965f-d78d90806c41 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:28.726965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9063de04-d740-44bc-965f-d78d90806c41 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:28.732306 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba533e-3502-4b2a-8476-ce577186701f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:28.732903 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba533e-3502-4b2a-8476-ce577186701f admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:28.733190 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba533e-3502-4b2a-8476-ce577186701f admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:28.733463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba533e-3502-4b2a-8476-ce577186701f admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:28.734096 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba533e-3502-4b2a-8476-ce577186701f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:28.735735 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba533e-3502-4b2a-8476-ce577186701f admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:28.746343 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1225/2446] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:15:28.750516 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03914a32-0b46-420b-9d88-48673ea5f485 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:28.761016 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03914a32-0b46-420b-9d88-48673ea5f485 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:28.765757 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff79627-45af-4a0f-b1a8-0bb1122ae4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:28.766592 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff79627-45af-4a0f-b1a8-0bb1122ae4b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:28.766903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff79627-45af-4a0f-b1a8-0bb1122ae4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:28.767178 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff79627-45af-4a0f-b1a8-0bb1122ae4b4 admin admin] PATH_INFO: `/v3/projects/98477d8315ec4a13bbef225588d7710c/users/dc066a127f89458ab05539e0a28872e6/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:28.782406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff79627-45af-4a0f-b1a8-0bb1122ae4b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98477d8315ec4a13bbef225588d7710c, user_id=dc066a127f89458ab05539e0a28872e6, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:28.785100 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff79627-45af-4a0f-b1a8-0bb1122ae4b4 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:28.798741 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1222/2447] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:15:28 2022] PUT /identity/v3/projects/98477d8315ec4a13bbef225588d7710c/users/dc066a127f89458ab05539e0a28872e6/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:15:28.810467 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d9e5b-17a0-4121-8685-b0fde641cf06 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:28.810759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d9e5b-17a0-4121-8685-b0fde641cf06 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:28.811036 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d9e5b-17a0-4121-8685-b0fde641cf06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:28.839356 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-f99d9e5b-17a0-4121-8685-b0fde641cf06 None None] MFA Rules not processed for user `dc066a127f89458ab05539e0a28872e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:15:28.840593 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f99d9e5b-17a0-4121-8685-b0fde641cf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:28.904196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1226/2448] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:15:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:15:28.915651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff79627-45af-4a0f-b1a8-0bb1122ae4b4 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:28.917395 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1ff79627-45af-4a0f-b1a8-0bb1122ae4b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:28.989974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff79627-45af-4a0f-b1a8-0bb1122ae4b4 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:28.996184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f314f4-f6d0-4096-933c-835ff3bff881 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:28.996785 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f314f4-f6d0-4096-933c-835ff3bff881 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:28.997065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f314f4-f6d0-4096-933c-835ff3bff881 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:28.997340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f314f4-f6d0-4096-933c-835ff3bff881 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:29.004985 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f314f4-f6d0-4096-933c-835ff3bff881 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:29.006810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f314f4-f6d0-4096-933c-835ff3bff881 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:29.015651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1223/2449] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:15:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:15:29.220402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d9e5b-17a0-4121-8685-b0fde641cf06 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:29.286002 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d9e5b-17a0-4121-8685-b0fde641cf06 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:29.291014 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf85a7a-16d0-493f-b91a-db36f7525567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:29.291666 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf85a7a-16d0-493f-b91a-db36f7525567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:29.291949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf85a7a-16d0-493f-b91a-db36f7525567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:29.292224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf85a7a-16d0-493f-b91a-db36f7525567 admin admin] PATH_INFO: `/v3/users/dc066a127f89458ab05539e0a28872e6` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:29.293332 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf85a7a-16d0-493f-b91a-db36f7525567 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc066a127f89458ab05539e0a28872e6)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:29.295252 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf85a7a-16d0-493f-b91a-db36f7525567 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:29.297608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:15:29.342287 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-abf85a7a-16d0-493f-b91a-db36f7525567 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc066a127f89458ab05539e0a28872e6'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:29.342919 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-abf85a7a-16d0-493f-b91a-db36f7525567 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc066a127f89458ab05539e0a28872e6'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:29.350095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-abf85a7a-16d0-493f-b91a-db36f7525567 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc066a127f89458ab05539e0a28872e6'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:29.358056 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-abf85a7a-16d0-493f-b91a-db36f7525567 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc066a127f89458ab05539e0a28872e6'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:29.358733 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-abf85a7a-16d0-493f-b91a-db36f7525567 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc066a127f89458ab05539e0a28872e6'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:29.376425 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1227/2450] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:15:29 2022] DELETE /identity/v3/users/dc066a127f89458ab05539e0a28872e6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:15:29.381220 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63f74c-b010-4390-9cc2-2f730325435f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:29.381518 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63f74c-b010-4390-9cc2-2f730325435f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:29.381795 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63f74c-b010-4390-9cc2-2f730325435f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:29.401312 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-fa63f74c-b010-4390-9cc2-2f730325435f None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:15:29.402539 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fa63f74c-b010-4390-9cc2-2f730325435f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:29.460062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1224/2451] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:15:29.470790 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf85a7a-16d0-493f-b91a-db36f7525567 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:29.472223 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-abf85a7a-16d0-493f-b91a-db36f7525567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:29.531072 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf85a7a-16d0-493f-b91a-db36f7525567 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:29.534737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624505a7-6cf2-4fdf-a562-1d190fb4a77d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:29.535342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624505a7-6cf2-4fdf-a562-1d190fb4a77d service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:29.535724 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624505a7-6cf2-4fdf-a562-1d190fb4a77d service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:29.536006 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624505a7-6cf2-4fdf-a562-1d190fb4a77d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:29.545517 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624505a7-6cf2-4fdf-a562-1d190fb4a77d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:29.546955 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624505a7-6cf2-4fdf-a562-1d190fb4a77d service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:29.556267 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1228/2452] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:15:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:15:29.835150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63f74c-b010-4390-9cc2-2f730325435f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:29.836749 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fa63f74c-b010-4390-9cc2-2f730325435f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:29.849205 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63f74c-b010-4390-9cc2-2f730325435f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:29.853969 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0419bc-f0bd-42fb-98ad-f5ee24d1fd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:29.854689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0419bc-f0bd-42fb-98ad-f5ee24d1fd93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:29.855043 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0419bc-f0bd-42fb-98ad-f5ee24d1fd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:29.855392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0419bc-f0bd-42fb-98ad-f5ee24d1fd93 admin admin] PATH_INFO: `/v3/projects/98477d8315ec4a13bbef225588d7710c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:29.856630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0419bc-f0bd-42fb-98ad-f5ee24d1fd93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98477d8315ec4a13bbef225588d7710c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:29.858694 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0419bc-f0bd-42fb-98ad-f5ee24d1fd93 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:29.892194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:15:29.909311 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2a0419bc-f0bd-42fb-98ad-f5ee24d1fd93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98477d8315ec4a13bbef225588d7710c', 'request_id': 'req-2a0419bc-f0bd-42fb-98ad-f5ee24d1fd93'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:29.916860 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2a0419bc-f0bd-42fb-98ad-f5ee24d1fd93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98477d8315ec4a13bbef225588d7710c', 'request_id': 'req-2a0419bc-f0bd-42fb-98ad-f5ee24d1fd93'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:29.922897 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2a0419bc-f0bd-42fb-98ad-f5ee24d1fd93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98477d8315ec4a13bbef225588d7710c', 'request_id': 'req-2a0419bc-f0bd-42fb-98ad-f5ee24d1fd93'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:29.934210 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2a0419bc-f0bd-42fb-98ad-f5ee24d1fd93 admin admin] The token cache is being invalidate because project 98477d8315ec4a13bbef225588d7710c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:15:29.934498 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2a0419bc-f0bd-42fb-98ad-f5ee24d1fd93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:29.935140 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2a0419bc-f0bd-42fb-98ad-f5ee24d1fd93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:29.936658 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1225/2453] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:15:29 2022] DELETE /identity/v3/projects/98477d8315ec4a13bbef225588d7710c => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:15:29.941474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebedbe-af36-435e-a8b1-210c7a407c76 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:29.941767 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebedbe-af36-435e-a8b1-210c7a407c76 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:29.942049 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebedbe-af36-435e-a8b1-210c7a407c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:29.962343 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-42ebedbe-af36-435e-a8b1-210c7a407c76 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:15:29.963538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-42ebedbe-af36-435e-a8b1-210c7a407c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:30.023411 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1229/2454] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:15:30.031808 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dad78-7227-475a-9c11-79d9642874c9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:30.032167 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dad78-7227-475a-9c11-79d9642874c9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:30.032513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dad78-7227-475a-9c11-79d9642874c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:30.052847 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-f22dad78-7227-475a-9c11-79d9642874c9 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:15:30.054036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f22dad78-7227-475a-9c11-79d9642874c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:30.062915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1226/2455] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:15:30.067123 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebedbe-af36-435e-a8b1-210c7a407c76 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:30.077224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebedbe-af36-435e-a8b1-210c7a407c76 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:30.081799 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bef8d-6a45-47ba-80a5-6ecbfb587c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:30.082392 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bef8d-6a45-47ba-80a5-6ecbfb587c85 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:30.082671 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bef8d-6a45-47ba-80a5-6ecbfb587c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:30.082942 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bef8d-6a45-47ba-80a5-6ecbfb587c85 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:30.083656 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7bef8d-6a45-47ba-80a5-6ecbfb587c85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:15:30.083944 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7bef8d-6a45-47ba-80a5-6ecbfb587c85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:30.085532 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7bef8d-6a45-47ba-80a5-6ecbfb587c85 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:30.097716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1230/2456] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:15:30.102013 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dad78-7227-475a-9c11-79d9642874c9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:30.111950 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dad78-7227-475a-9c11-79d9642874c9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:30.116556 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c423e-551c-4fa1-a532-07f0aa590f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:30.117145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c423e-551c-4fa1-a532-07f0aa590f88 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:30.117421 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c423e-551c-4fa1-a532-07f0aa590f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:30.117690 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c423e-551c-4fa1-a532-07f0aa590f88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:30.118438 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c423e-551c-4fa1-a532-07f0aa590f88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:30.120367 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c423e-551c-4fa1-a532-07f0aa590f88 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:30.148563 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1227/2457] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:15:30 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:15:30.153156 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bef8d-6a45-47ba-80a5-6ecbfb587c85 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:30.162884 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bef8d-6a45-47ba-80a5-6ecbfb587c85 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:30.173319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:15:30.226522 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27cb70-e416-4a32-a4ca-1c8b8227f4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:30.227165 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27cb70-e416-4a32-a4ca-1c8b8227f4eb admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:30.228074 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27cb70-e416-4a32-a4ca-1c8b8227f4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:30.228342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27cb70-e416-4a32-a4ca-1c8b8227f4eb admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:30.228988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf27cb70-e416-4a32-a4ca-1c8b8227f4eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:30.230928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf27cb70-e416-4a32-a4ca-1c8b8227f4eb admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:30.251929 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1231/2458] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:15:30 2022] POST /identity/v3/users => generated 303 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:15:30.256361 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c423e-551c-4fa1-a532-07f0aa590f88 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:30.267236 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c423e-551c-4fa1-a532-07f0aa590f88 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:30.272120 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46b276-c68e-487a-a5ea-e82a471fca33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:30.272716 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46b276-c68e-487a-a5ea-e82a471fca33 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:30.272996 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46b276-c68e-487a-a5ea-e82a471fca33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:30.273269 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46b276-c68e-487a-a5ea-e82a471fca33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:30.274055 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46b276-c68e-487a-a5ea-e82a471fca33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:30.276542 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46b276-c68e-487a-a5ea-e82a471fca33 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:30.286331 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1228/2459] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:15:30.291154 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27cb70-e416-4a32-a4ca-1c8b8227f4eb admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:30.301051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27cb70-e416-4a32-a4ca-1c8b8227f4eb admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:30.305695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937fcf4-e603-4d9f-b477-4405db670d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:30.306286 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937fcf4-e603-4d9f-b477-4405db670d58 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:30.306566 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937fcf4-e603-4d9f-b477-4405db670d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:30.306841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937fcf4-e603-4d9f-b477-4405db670d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:30.307478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e937fcf4-e603-4d9f-b477-4405db670d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:30.309095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e937fcf4-e603-4d9f-b477-4405db670d58 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:30.319507 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1232/2460] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:15:30.323813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46b276-c68e-487a-a5ea-e82a471fca33 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:30.333508 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46b276-c68e-487a-a5ea-e82a471fca33 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:30.338573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d2bf2-8fd8-49b5-816e-25c1328aa0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:30.339208 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d2bf2-8fd8-49b5-816e-25c1328aa0b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:30.339486 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d2bf2-8fd8-49b5-816e-25c1328aa0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:30.339802 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d2bf2-8fd8-49b5-816e-25c1328aa0b3 admin admin] PATH_INFO: `/v3/projects/7f9d3449c9c94a93a787e4aed5c667a7/users/1b863c4d1d2448daad87f8e54a30e40d/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:30.354548 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d2bf2-8fd8-49b5-816e-25c1328aa0b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f9d3449c9c94a93a787e4aed5c667a7, user_id=1b863c4d1d2448daad87f8e54a30e40d, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:30.357215 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d2bf2-8fd8-49b5-816e-25c1328aa0b3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:30.369769 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1229/2461] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:15:30 2022] PUT /identity/v3/projects/7f9d3449c9c94a93a787e4aed5c667a7/users/1b863c4d1d2448daad87f8e54a30e40d/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:15:30.381551 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41707f73-c508-4b68-beec-63dbcd5a80c9 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:30.381840 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41707f73-c508-4b68-beec-63dbcd5a80c9 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:30.382111 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41707f73-c508-4b68-beec-63dbcd5a80c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:30.410426 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-41707f73-c508-4b68-beec-63dbcd5a80c9 None None] MFA Rules not processed for user `1b863c4d1d2448daad87f8e54a30e40d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:15:30.411673 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-41707f73-c508-4b68-beec-63dbcd5a80c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:30.477694 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1233/2462] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:15:30 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:15:30.484727 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d2bf2-8fd8-49b5-816e-25c1328aa0b3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:30.546697 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d2bf2-8fd8-49b5-816e-25c1328aa0b3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:30.551247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5fb33-a221-4bd0-af84-32c34ade08b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:30.551884 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5fb33-a221-4bd0-af84-32c34ade08b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:30.552163 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5fb33-a221-4bd0-af84-32c34ade08b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:30.552437 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5fb33-a221-4bd0-af84-32c34ade08b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:30.553060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb5fb33-a221-4bd0-af84-32c34ade08b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:30.554957 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb5fb33-a221-4bd0-af84-32c34ade08b7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:30.581866 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1230/2463] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:15:30 2022] POST /identity/v3/projects => generated 405 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:15:30.586035 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41707f73-c508-4b68-beec-63dbcd5a80c9 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:30.597385 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41707f73-c508-4b68-beec-63dbcd5a80c9 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:30.658116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49472bc9-d553-432f-a994-eeb00ba9c169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:30.658728 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49472bc9-d553-432f-a994-eeb00ba9c169 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:30.659081 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49472bc9-d553-432f-a994-eeb00ba9c169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:30.659307 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49472bc9-d553-432f-a994-eeb00ba9c169 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:30.660014 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49472bc9-d553-432f-a994-eeb00ba9c169 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:30.662138 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49472bc9-d553-432f-a994-eeb00ba9c169 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:30.684842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1234/2464] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:15:30 2022] POST /identity/v3/users => generated 309 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:15:30.689017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5fb33-a221-4bd0-af84-32c34ade08b7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:30.700270 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5fb33-a221-4bd0-af84-32c34ade08b7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:30.705186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ab094-06f5-48d2-badf-cd75a156eea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:30.705790 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ab094-06f5-48d2-badf-cd75a156eea4 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:30.706068 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ab094-06f5-48d2-badf-cd75a156eea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:30.706341 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ab094-06f5-48d2-badf-cd75a156eea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:30.706982 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ab094-06f5-48d2-badf-cd75a156eea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:30.708633 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ab094-06f5-48d2-badf-cd75a156eea4 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:30.723582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1231/2465] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:15:30.724779 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49472bc9-d553-432f-a994-eeb00ba9c169 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:30.735059 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49472bc9-d553-432f-a994-eeb00ba9c169 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:30.740131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c53c49-b32a-4fb5-88df-e95313b4018d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:30.740792 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c53c49-b32a-4fb5-88df-e95313b4018d admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:30.741078 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c53c49-b32a-4fb5-88df-e95313b4018d admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:30.741357 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c53c49-b32a-4fb5-88df-e95313b4018d admin admin] PATH_INFO: `/v3/projects/1e805b6851db47a59589a1feb2b75ac3/users/968c1e164c654b8aa6420ddbd2e57b0e/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:30.744871 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:15:30.768583 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c53c49-b32a-4fb5-88df-e95313b4018d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e805b6851db47a59589a1feb2b75ac3, user_id=968c1e164c654b8aa6420ddbd2e57b0e, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:30.771171 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c53c49-b32a-4fb5-88df-e95313b4018d admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:30.787336 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1235/2466] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:15:30 2022] PUT /identity/v3/projects/1e805b6851db47a59589a1feb2b75ac3/users/968c1e164c654b8aa6420ddbd2e57b0e/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:15:30.791452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ab094-06f5-48d2-badf-cd75a156eea4 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:30.801787 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ab094-06f5-48d2-badf-cd75a156eea4 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:30.839288 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:15:30.875026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c53c49-b32a-4fb5-88df-e95313b4018d admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:30.877657 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-61c53c49-b32a-4fb5-88df-e95313b4018d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:30.879323 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6267d45-b6a6-4b22-9159-f8408fb304bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:30.879859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6267d45-b6a6-4b22-9159-f8408fb304bf admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:30.880152 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6267d45-b6a6-4b22-9159-f8408fb304bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:30.880844 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6267d45-b6a6-4b22-9159-f8408fb304bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:30.881322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6267d45-b6a6-4b22-9159-f8408fb304bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:30.883651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6267d45-b6a6-4b22-9159-f8408fb304bf admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:30.895006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1232/2467] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:15:30.899243 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6267d45-b6a6-4b22-9159-f8408fb304bf admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:30.910532 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6267d45-b6a6-4b22-9159-f8408fb304bf admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:30.917519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67ae1b-74e7-43a7-a9f8-c937820b5e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:30.919102 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67ae1b-74e7-43a7-a9f8-c937820b5e47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:30.919102 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67ae1b-74e7-43a7-a9f8-c937820b5e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:30.919102 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67ae1b-74e7-43a7-a9f8-c937820b5e47 admin admin] PATH_INFO: `/v3/projects/1e805b6851db47a59589a1feb2b75ac3/users/968c1e164c654b8aa6420ddbd2e57b0e/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:30.920737 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a67ae1b-74e7-43a7-a9f8-c937820b5e47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e805b6851db47a59589a1feb2b75ac3, user_id=968c1e164c654b8aa6420ddbd2e57b0e, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:30.924605 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a67ae1b-74e7-43a7-a9f8-c937820b5e47 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:30.938619 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1233/2468] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:15:30 2022] PUT /identity/v3/projects/1e805b6851db47a59589a1feb2b75ac3/users/968c1e164c654b8aa6420ddbd2e57b0e/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:15:30.947229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c53c49-b32a-4fb5-88df-e95313b4018d admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:30.949850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313975d-390f-497d-99b5-d3df0e6f3e4f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:30.950431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313975d-390f-497d-99b5-d3df0e6f3e4f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:30.950794 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313975d-390f-497d-99b5-d3df0e6f3e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:30.987450 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d313975d-390f-497d-99b5-d3df0e6f3e4f None None] MFA Rules not processed for user `968c1e164c654b8aa6420ddbd2e57b0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:15:30.988683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d313975d-390f-497d-99b5-d3df0e6f3e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:31.002023 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad0a49-cf48-4b4d-9ccb-fdd1699bb716 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:31.002637 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad0a49-cf48-4b4d-9ccb-fdd1699bb716 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:31.002917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad0a49-cf48-4b4d-9ccb-fdd1699bb716 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:31.003194 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad0a49-cf48-4b4d-9ccb-fdd1699bb716 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:31.005070 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-50ad0a49-cf48-4b4d-9ccb-fdd1699bb716 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:31.015680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad0a49-cf48-4b4d-9ccb-fdd1699bb716 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:31.017108 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad0a49-cf48-4b4d-9ccb-fdd1699bb716 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:31.025143 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1236/2469] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:15:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:15:31.060117 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1234/2470] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:15:30 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:15:31.070914 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad0a49-cf48-4b4d-9ccb-fdd1699bb716 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:31.073256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-50ad0a49-cf48-4b4d-9ccb-fdd1699bb716 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:31.130899 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad0a49-cf48-4b4d-9ccb-fdd1699bb716 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:31.135915 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e0503-6689-4630-87de-753b6d8e46ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:31.136518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e0503-6689-4630-87de-753b6d8e46ac service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:31.136797 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e0503-6689-4630-87de-753b6d8e46ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:31.137073 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e0503-6689-4630-87de-753b6d8e46ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:31.146210 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e0503-6689-4630-87de-753b6d8e46ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:31.147682 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e0503-6689-4630-87de-753b6d8e46ac service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:31.157735 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1237/2471] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:15:31 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:15:31.312165 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d313975d-390f-497d-99b5-d3df0e6f3e4f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:31.377169 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d313975d-390f-497d-99b5-d3df0e6f3e4f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:31.381972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ad76f-6e64-45a2-9d4d-64224a0b2f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:31.382602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ad76f-6e64-45a2-9d4d-64224a0b2f2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:31.382909 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ad76f-6e64-45a2-9d4d-64224a0b2f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:31.383202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ad76f-6e64-45a2-9d4d-64224a0b2f2f admin admin] PATH_INFO: `/v3/users/1b863c4d1d2448daad87f8e54a30e40d` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:31.384421 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ad76f-6e64-45a2-9d4d-64224a0b2f2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b863c4d1d2448daad87f8e54a30e40d)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:31.386365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ad76f-6e64-45a2-9d4d-64224a0b2f2f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:31.426268 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:15:31.438711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c08ad76f-6e64-45a2-9d4d-64224a0b2f2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b863c4d1d2448daad87f8e54a30e40d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:31.439499 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c08ad76f-6e64-45a2-9d4d-64224a0b2f2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b863c4d1d2448daad87f8e54a30e40d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:31.447527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c08ad76f-6e64-45a2-9d4d-64224a0b2f2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b863c4d1d2448daad87f8e54a30e40d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:31.448179 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c08ad76f-6e64-45a2-9d4d-64224a0b2f2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b863c4d1d2448daad87f8e54a30e40d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:31.455226 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c08ad76f-6e64-45a2-9d4d-64224a0b2f2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b863c4d1d2448daad87f8e54a30e40d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:31.474287 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1235/2472] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:15:31 2022] DELETE /identity/v3/users/1b863c4d1d2448daad87f8e54a30e40d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:15:31.479617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3cee4-85df-457c-900e-49ca6cb8a0d0 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:31.480139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3cee4-85df-457c-900e-49ca6cb8a0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:31.480229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3cee4-85df-457c-900e-49ca6cb8a0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:31.499722 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-fdc3cee4-85df-457c-900e-49ca6cb8a0d0 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:15:31.500930 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-fdc3cee4-85df-457c-900e-49ca6cb8a0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:31.544423 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:15:31.563859 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1238/2473] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:15:31.573787 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ad76f-6e64-45a2-9d4d-64224a0b2f2f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:31.575193 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c08ad76f-6e64-45a2-9d4d-64224a0b2f2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:31.631683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ad76f-6e64-45a2-9d4d-64224a0b2f2f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:31.635699 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88042e1e-15ea-40cf-9665-642abd40452c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:31.636313 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88042e1e-15ea-40cf-9665-642abd40452c service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:31.636595 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88042e1e-15ea-40cf-9665-642abd40452c service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:31.636869 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88042e1e-15ea-40cf-9665-642abd40452c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:31.646370 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88042e1e-15ea-40cf-9665-642abd40452c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:31.647846 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88042e1e-15ea-40cf-9665-642abd40452c service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:31.662167 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1236/2474] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:15:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:15:31.943606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3cee4-85df-457c-900e-49ca6cb8a0d0 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:31.945177 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-fdc3cee4-85df-457c-900e-49ca6cb8a0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:31.960323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3cee4-85df-457c-900e-49ca6cb8a0d0 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:31.966633 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80372a5d-2659-406b-bc56-ee41a684dc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:31.967281 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80372a5d-2659-406b-bc56-ee41a684dc29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:31.967769 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80372a5d-2659-406b-bc56-ee41a684dc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:31.968051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80372a5d-2659-406b-bc56-ee41a684dc29 admin admin] PATH_INFO: `/v3/projects/7f9d3449c9c94a93a787e4aed5c667a7` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:31.969350 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80372a5d-2659-406b-bc56-ee41a684dc29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f9d3449c9c94a93a787e4aed5c667a7)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:31.971273 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80372a5d-2659-406b-bc56-ee41a684dc29 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:32.023027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-80372a5d-2659-406b-bc56-ee41a684dc29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f9d3449c9c94a93a787e4aed5c667a7', 'request_id': 'req-80372a5d-2659-406b-bc56-ee41a684dc29'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:32.031692 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-80372a5d-2659-406b-bc56-ee41a684dc29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f9d3449c9c94a93a787e4aed5c667a7', 'request_id': 'req-80372a5d-2659-406b-bc56-ee41a684dc29'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:32.037678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-80372a5d-2659-406b-bc56-ee41a684dc29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f9d3449c9c94a93a787e4aed5c667a7', 'request_id': 'req-80372a5d-2659-406b-bc56-ee41a684dc29'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:32.049312 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-80372a5d-2659-406b-bc56-ee41a684dc29 admin admin] The token cache is being invalidate because project 7f9d3449c9c94a93a787e4aed5c667a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:15:32.049599 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-80372a5d-2659-406b-bc56-ee41a684dc29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:32.050393 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-80372a5d-2659-406b-bc56-ee41a684dc29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:32.051998 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1239/2475] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:15:31 2022] DELETE /identity/v3/projects/7f9d3449c9c94a93a787e4aed5c667a7 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:15:32.056752 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88042e1e-15ea-40cf-9665-642abd40452c service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:32.066630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88042e1e-15ea-40cf-9665-642abd40452c service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:32.125913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60ee2f-3f55-4eb7-914d-6df57a4573e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:32.126561 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60ee2f-3f55-4eb7-914d-6df57a4573e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:32.126833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60ee2f-3f55-4eb7-914d-6df57a4573e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:32.127132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60ee2f-3f55-4eb7-914d-6df57a4573e5 admin admin] PATH_INFO: `/v3/users/968c1e164c654b8aa6420ddbd2e57b0e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:32.128373 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e60ee2f-3f55-4eb7-914d-6df57a4573e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=968c1e164c654b8aa6420ddbd2e57b0e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:32.130313 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e60ee2f-3f55-4eb7-914d-6df57a4573e5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:32.183249 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4e60ee2f-3f55-4eb7-914d-6df57a4573e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '968c1e164c654b8aa6420ddbd2e57b0e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:32.184013 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4e60ee2f-3f55-4eb7-914d-6df57a4573e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '968c1e164c654b8aa6420ddbd2e57b0e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:32.191232 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4e60ee2f-3f55-4eb7-914d-6df57a4573e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '968c1e164c654b8aa6420ddbd2e57b0e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:32.191853 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4e60ee2f-3f55-4eb7-914d-6df57a4573e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '968c1e164c654b8aa6420ddbd2e57b0e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:32.196932 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:15:32.200965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-4e60ee2f-3f55-4eb7-914d-6df57a4573e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '968c1e164c654b8aa6420ddbd2e57b0e'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:32.219669 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1237/2476] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:15:32 2022] DELETE /identity/v3/users/968c1e164c654b8aa6420ddbd2e57b0e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:15:32.489893 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80372a5d-2659-406b-bc56-ee41a684dc29 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:32.491375 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-80372a5d-2659-406b-bc56-ee41a684dc29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:32.504954 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80372a5d-2659-406b-bc56-ee41a684dc29 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:32.562782 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64054517-1ce6-4f3c-8dd3-c01437dd606a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:32.563406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64054517-1ce6-4f3c-8dd3-c01437dd606a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:32.563716 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64054517-1ce6-4f3c-8dd3-c01437dd606a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:32.563992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64054517-1ce6-4f3c-8dd3-c01437dd606a admin admin] PATH_INFO: `/v3/projects/1e805b6851db47a59589a1feb2b75ac3` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:32.565096 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64054517-1ce6-4f3c-8dd3-c01437dd606a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e805b6851db47a59589a1feb2b75ac3)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:32.566992 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64054517-1ce6-4f3c-8dd3-c01437dd606a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:32.611754 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-64054517-1ce6-4f3c-8dd3-c01437dd606a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e805b6851db47a59589a1feb2b75ac3', 'request_id': 'req-64054517-1ce6-4f3c-8dd3-c01437dd606a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:32.619528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-64054517-1ce6-4f3c-8dd3-c01437dd606a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e805b6851db47a59589a1feb2b75ac3', 'request_id': 'req-64054517-1ce6-4f3c-8dd3-c01437dd606a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:32.625109 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-64054517-1ce6-4f3c-8dd3-c01437dd606a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e805b6851db47a59589a1feb2b75ac3', 'request_id': 'req-64054517-1ce6-4f3c-8dd3-c01437dd606a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:32.632949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:15:32.637913 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-64054517-1ce6-4f3c-8dd3-c01437dd606a admin admin] The token cache is being invalidate because project 1e805b6851db47a59589a1feb2b75ac3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:15:32.638412 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-64054517-1ce6-4f3c-8dd3-c01437dd606a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:32.639110 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-64054517-1ce6-4f3c-8dd3-c01437dd606a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:15:32.641113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1240/2477] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:15:32 2022] DELETE /identity/v3/projects/1e805b6851db47a59589a1feb2b75ac3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:15:32.646185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd12a60-790e-4a4b-bd2a-9b72221b3811 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:32.646470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd12a60-790e-4a4b-bd2a-9b72221b3811 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:32.646768 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd12a60-790e-4a4b-bd2a-9b72221b3811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:32.667468 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-bdd12a60-790e-4a4b-bd2a-9b72221b3811 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:15:32.668704 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bdd12a60-790e-4a4b-bd2a-9b72221b3811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:32.727090 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1238/2478] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:15:32.735584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26bc9b8-9475-46c0-8cdb-6061b52dedf3 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:32.735844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26bc9b8-9475-46c0-8cdb-6061b52dedf3 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:32.736130 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26bc9b8-9475-46c0-8cdb-6061b52dedf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:32.756543 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a26bc9b8-9475-46c0-8cdb-6061b52dedf3 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:15:32.757735 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a26bc9b8-9475-46c0-8cdb-6061b52dedf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:32.765656 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1241/2479] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:15:32.769787 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd12a60-790e-4a4b-bd2a-9b72221b3811 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:32.779808 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd12a60-790e-4a4b-bd2a-9b72221b3811 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:32.784383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192a84d-b8b3-4fe9-aebf-6049de7b4318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:32.784977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192a84d-b8b3-4fe9-aebf-6049de7b4318 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:32.785254 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192a84d-b8b3-4fe9-aebf-6049de7b4318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:32.785521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192a84d-b8b3-4fe9-aebf-6049de7b4318 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:32.786186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192a84d-b8b3-4fe9-aebf-6049de7b4318 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:15:32.786518 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192a84d-b8b3-4fe9-aebf-6049de7b4318 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:32.788151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192a84d-b8b3-4fe9-aebf-6049de7b4318 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:32.800281 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1239/2480] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:15:32.804431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26bc9b8-9475-46c0-8cdb-6061b52dedf3 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:32.814540 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26bc9b8-9475-46c0-8cdb-6061b52dedf3 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:32.819091 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4654a-0eea-4257-b03d-d4a1a40cf630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:32.819721 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4654a-0eea-4257-b03d-d4a1a40cf630 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:32.819999 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4654a-0eea-4257-b03d-d4a1a40cf630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:32.820270 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4654a-0eea-4257-b03d-d4a1a40cf630 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:32.820903 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4654a-0eea-4257-b03d-d4a1a40cf630 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:32.823156 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4654a-0eea-4257-b03d-d4a1a40cf630 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:32.849279 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1242/2481] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:15:32 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:15:32.853984 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192a84d-b8b3-4fe9-aebf-6049de7b4318 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:32.863797 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192a84d-b8b3-4fe9-aebf-6049de7b4318 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:32.920020 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb36930-7e46-4890-86fe-3e5f01fe72d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:32.920625 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb36930-7e46-4890-86fe-3e5f01fe72d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:32.920935 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb36930-7e46-4890-86fe-3e5f01fe72d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:32.921210 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb36930-7e46-4890-86fe-3e5f01fe72d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:32.921848 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb36930-7e46-4890-86fe-3e5f01fe72d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:32.923787 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb36930-7e46-4890-86fe-3e5f01fe72d3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:32.946917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1240/2482] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:15:32 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:15:32.951657 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4654a-0eea-4257-b03d-d4a1a40cf630 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:32.962047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4654a-0eea-4257-b03d-d4a1a40cf630 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:32.966689 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a015c40-3fc3-452c-b9dd-1a32aaa04205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:32.967290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a015c40-3fc3-452c-b9dd-1a32aaa04205 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:32.967606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a015c40-3fc3-452c-b9dd-1a32aaa04205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:32.967893 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a015c40-3fc3-452c-b9dd-1a32aaa04205 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:32.968565 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a015c40-3fc3-452c-b9dd-1a32aaa04205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:32.970183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a015c40-3fc3-452c-b9dd-1a32aaa04205 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:32.980776 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1243/2483] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:15:32.984734 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb36930-7e46-4890-86fe-3e5f01fe72d3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:32.995585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb36930-7e46-4890-86fe-3e5f01fe72d3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:33.000099 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfad42f-fe13-4aab-a637-b3387e01bc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:33.000687 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfad42f-fe13-4aab-a637-b3387e01bc55 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:33.000966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfad42f-fe13-4aab-a637-b3387e01bc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:33.001237 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfad42f-fe13-4aab-a637-b3387e01bc55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:33.001871 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfad42f-fe13-4aab-a637-b3387e01bc55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:33.003477 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfad42f-fe13-4aab-a637-b3387e01bc55 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:33.014311 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1241/2484] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:15:33.018332 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a015c40-3fc3-452c-b9dd-1a32aaa04205 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:33.028436 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a015c40-3fc3-452c-b9dd-1a32aaa04205 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:33.033016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0530e9f-927e-45c4-b039-3cc2d2892643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:33.033661 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0530e9f-927e-45c4-b039-3cc2d2892643 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:33.033943 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0530e9f-927e-45c4-b039-3cc2d2892643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:33.034219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0530e9f-927e-45c4-b039-3cc2d2892643 admin admin] PATH_INFO: `/v3/projects/336c8692bb0649a5829ff643f5d18bb3/users/42db972b0209451f82f7783eb5fdc5fd/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:33.049220 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0530e9f-927e-45c4-b039-3cc2d2892643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=336c8692bb0649a5829ff643f5d18bb3, user_id=42db972b0209451f82f7783eb5fdc5fd, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:33.051925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0530e9f-927e-45c4-b039-3cc2d2892643 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:33.065538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1244/2485] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:15:33 2022] PUT /identity/v3/projects/336c8692bb0649a5829ff643f5d18bb3/users/42db972b0209451f82f7783eb5fdc5fd/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:15:33.077274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf811bba-c234-47b1-9b7f-8cd1e523550c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:33.077800 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf811bba-c234-47b1-9b7f-8cd1e523550c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:33.077914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf811bba-c234-47b1-9b7f-8cd1e523550c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:33.106338 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-cf811bba-c234-47b1-9b7f-8cd1e523550c None None] MFA Rules not processed for user `42db972b0209451f82f7783eb5fdc5fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:15:33.107545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cf811bba-c234-47b1-9b7f-8cd1e523550c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:33.137173 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:15:33.174231 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1242/2486] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:15:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:15:33.178775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0530e9f-927e-45c4-b039-3cc2d2892643 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:33.243651 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0530e9f-927e-45c4-b039-3cc2d2892643 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:33.248659 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d542d-5766-455e-bac3-33b56eb1e063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:33.249206 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d542d-5766-455e-bac3-33b56eb1e063 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:33.249481 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d542d-5766-455e-bac3-33b56eb1e063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:33.249749 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d542d-5766-455e-bac3-33b56eb1e063 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:33.250379 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d542d-5766-455e-bac3-33b56eb1e063 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:33.252318 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d542d-5766-455e-bac3-33b56eb1e063 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:33.280188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1245/2487] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:15:33 2022] POST /identity/v3/projects => generated 371 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:15:33.284304 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf811bba-c234-47b1-9b7f-8cd1e523550c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:33.295951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf811bba-c234-47b1-9b7f-8cd1e523550c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:33.352855 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f355b-c70e-454e-9661-af46cb169ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:33.353467 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f355b-c70e-454e-9661-af46cb169ef8 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:33.353744 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f355b-c70e-454e-9661-af46cb169ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:33.354021 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f355b-c70e-454e-9661-af46cb169ef8 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:33.354685 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f355b-c70e-454e-9661-af46cb169ef8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:33.356692 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f355b-c70e-454e-9661-af46cb169ef8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:33.378175 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1243/2488] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:15:33 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:15:33.382378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d542d-5766-455e-bac3-33b56eb1e063 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:33.393883 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d542d-5766-455e-bac3-33b56eb1e063 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:33.398393 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559a986-e57f-41f4-a4b1-a0e4e496885b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:33.398985 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559a986-e57f-41f4-a4b1-a0e4e496885b admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:33.399350 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559a986-e57f-41f4-a4b1-a0e4e496885b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:33.399895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559a986-e57f-41f4-a4b1-a0e4e496885b admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:33.400565 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3559a986-e57f-41f4-a4b1-a0e4e496885b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:33.402188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3559a986-e57f-41f4-a4b1-a0e4e496885b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:33.413085 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1246/2489] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:15:33.417349 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f355b-c70e-454e-9661-af46cb169ef8 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:33.427230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f355b-c70e-454e-9661-af46cb169ef8 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:33.432799 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28ee0a-f73c-4465-9f35-aada58f12c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:33.433446 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28ee0a-f73c-4465-9f35-aada58f12c6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:33.433726 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28ee0a-f73c-4465-9f35-aada58f12c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:33.434028 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28ee0a-f73c-4465-9f35-aada58f12c6e admin admin] PATH_INFO: `/v3/projects/c5f46d9a4cc74c96a33743381f38b69d/users/6a6c01904d33403f9b8a032ee749f9dc/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:33.449432 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c28ee0a-f73c-4465-9f35-aada58f12c6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5f46d9a4cc74c96a33743381f38b69d, user_id=6a6c01904d33403f9b8a032ee749f9dc, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:33.452111 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c28ee0a-f73c-4465-9f35-aada58f12c6e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:33.464578 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1244/2490] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:15:33 2022] PUT /identity/v3/projects/c5f46d9a4cc74c96a33743381f38b69d/users/6a6c01904d33403f9b8a032ee749f9dc/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:15:33.468570 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559a986-e57f-41f4-a4b1-a0e4e496885b admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:33.479413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559a986-e57f-41f4-a4b1-a0e4e496885b admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:33.535376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f81a7e-f48c-4959-8514-b1d487e087b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:33.536025 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f81a7e-f48c-4959-8514-b1d487e087b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:33.536208 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f81a7e-f48c-4959-8514-b1d487e087b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:33.536485 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f81a7e-f48c-4959-8514-b1d487e087b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:33.537121 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f81a7e-f48c-4959-8514-b1d487e087b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:33.538726 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f81a7e-f48c-4959-8514-b1d487e087b4 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:33.548962 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1247/2491] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:15:33.553152 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28ee0a-f73c-4465-9f35-aada58f12c6e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:33.563881 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28ee0a-f73c-4465-9f35-aada58f12c6e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:33.568466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58dc45a-a710-4836-a72b-678cd415aeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:33.569106 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58dc45a-a710-4836-a72b-678cd415aeb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:33.569384 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58dc45a-a710-4836-a72b-678cd415aeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:33.569655 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58dc45a-a710-4836-a72b-678cd415aeb0 admin admin] PATH_INFO: `/v3/projects/c5f46d9a4cc74c96a33743381f38b69d/users/6a6c01904d33403f9b8a032ee749f9dc/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:33.571645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58dc45a-a710-4836-a72b-678cd415aeb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5f46d9a4cc74c96a33743381f38b69d, user_id=6a6c01904d33403f9b8a032ee749f9dc, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:33.574282 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58dc45a-a710-4836-a72b-678cd415aeb0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:33.584719 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1245/2492] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:15:33 2022] PUT /identity/v3/projects/c5f46d9a4cc74c96a33743381f38b69d/users/6a6c01904d33403f9b8a032ee749f9dc/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:15:33.594931 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41dad9-dfde-44a1-ba51-7131bde98c48 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:33.595220 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41dad9-dfde-44a1-ba51-7131bde98c48 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:33.595501 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41dad9-dfde-44a1-ba51-7131bde98c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:33.609965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:15:33.625302 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-ae41dad9-dfde-44a1-ba51-7131bde98c48 None None] MFA Rules not processed for user `6a6c01904d33403f9b8a032ee749f9dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:15:33.626521 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ae41dad9-dfde-44a1-ba51-7131bde98c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:33.700677 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1248/2493] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:15:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:15:33.711861 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58dc45a-a710-4836-a72b-678cd415aeb0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:33.713814 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d58dc45a-a710-4836-a72b-678cd415aeb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:33.771118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58dc45a-a710-4836-a72b-678cd415aeb0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:33.775010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ae83f-4707-4fe6-b33b-39283b876b4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:33.775638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ae83f-4707-4fe6-b33b-39283b876b4c service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:33.775916 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ae83f-4707-4fe6-b33b-39283b876b4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:33.776185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ae83f-4707-4fe6-b33b-39283b876b4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:33.784958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ae83f-4707-4fe6-b33b-39283b876b4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:33.786428 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ae83f-4707-4fe6-b33b-39283b876b4c service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:33.796340 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1246/2494] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:15:33 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:15:33.874024 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41dad9-dfde-44a1-ba51-7131bde98c48 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:33.883967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41dad9-dfde-44a1-ba51-7131bde98c48 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:33.887572 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67986c77-5709-4f8d-9b89-2e291b4b9d4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:33.888182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67986c77-5709-4f8d-9b89-2e291b4b9d4f service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:33.888466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67986c77-5709-4f8d-9b89-2e291b4b9d4f service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:33.888743 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67986c77-5709-4f8d-9b89-2e291b4b9d4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:33.944936 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67986c77-5709-4f8d-9b89-2e291b4b9d4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:33.946570 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67986c77-5709-4f8d-9b89-2e291b4b9d4f service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:33.955443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1249/2495] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:15:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:15:43.792180 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=83,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:15:43.951390 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=59 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:15:50.928575 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ae83f-4707-4fe6-b33b-39283b876b4c service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:15:50.930053 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-741ae83f-4707-4fe6-b33b-39283b876b4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:51.002603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ae83f-4707-4fe6-b33b-39283b876b4c service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:15:51.008294 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9389a-b1b3-471f-96df-850332c5b062 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:15:51.008898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9389a-b1b3-471f-96df-850332c5b062 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:15:51.009182 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9389a-b1b3-471f-96df-850332c5b062 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:15:51.009455 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9389a-b1b3-471f-96df-850332c5b062 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:15:51.011148 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-44b9389a-b1b3-471f-96df-850332c5b062 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:15:51.091097 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9389a-b1b3-471f-96df-850332c5b062 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:15:51.092666 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9389a-b1b3-471f-96df-850332c5b062 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:15:51.110738 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1247/2496] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:15:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:01.108284 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=69 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:19.383135 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67986c77-5709-4f8d-9b89-2e291b4b9d4f service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:19.384580 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-67986c77-5709-4f8d-9b89-2e291b4b9d4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:19.443104 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67986c77-5709-4f8d-9b89-2e291b4b9d4f service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:19.446707 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224cf8b5-575b-4569-aab8-1cabdbfa482f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:19.447316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224cf8b5-575b-4569-aab8-1cabdbfa482f service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:19.447620 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224cf8b5-575b-4569-aab8-1cabdbfa482f service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:19.447895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224cf8b5-575b-4569-aab8-1cabdbfa482f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:19.449778 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-224cf8b5-575b-4569-aab8-1cabdbfa482f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:19.523771 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224cf8b5-575b-4569-aab8-1cabdbfa482f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:19.525284 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224cf8b5-575b-4569-aab8-1cabdbfa482f service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:19.538651 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1250/2497] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:19.969268 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9389a-b1b3-471f-96df-850332c5b062 service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:19.979965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9389a-b1b3-471f-96df-850332c5b062 service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:19.988090 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ca628-50fa-45b6-9f33-88580d0cafdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:19.988745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ca628-50fa-45b6-9f33-88580d0cafdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:19.989027 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ca628-50fa-45b6-9f33-88580d0cafdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:19.989300 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ca628-50fa-45b6-9f33-88580d0cafdb admin admin] PATH_INFO: `/v3/users/42db972b0209451f82f7783eb5fdc5fd` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:19.992314 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ca628-50fa-45b6-9f33-88580d0cafdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=42db972b0209451f82f7783eb5fdc5fd)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:20.000833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ca628-50fa-45b6-9f33-88580d0cafdb admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:20.079139 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2e9ca628-50fa-45b6-9f33-88580d0cafdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42db972b0209451f82f7783eb5fdc5fd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:20.080138 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2e9ca628-50fa-45b6-9f33-88580d0cafdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42db972b0209451f82f7783eb5fdc5fd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:20.105504 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2e9ca628-50fa-45b6-9f33-88580d0cafdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42db972b0209451f82f7783eb5fdc5fd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:20.116266 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2e9ca628-50fa-45b6-9f33-88580d0cafdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42db972b0209451f82f7783eb5fdc5fd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:20.128924 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2e9ca628-50fa-45b6-9f33-88580d0cafdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42db972b0209451f82f7783eb5fdc5fd'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:20.156218 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1248/2498] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:19 2022] DELETE /identity/v3/users/42db972b0209451f82f7783eb5fdc5fd => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:20.160621 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab3bf5-c10e-4a77-942a-2aeedc3b2f87 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:20.161008 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab3bf5-c10e-4a77-942a-2aeedc3b2f87 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:20.161330 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab3bf5-c10e-4a77-942a-2aeedc3b2f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:20.184579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-2fab3bf5-c10e-4a77-942a-2aeedc3b2f87 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:20.186091 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2fab3bf5-c10e-4a77-942a-2aeedc3b2f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:20.276919 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1251/2499] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:20.288412 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ca628-50fa-45b6-9f33-88580d0cafdb admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:20.289825 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2e9ca628-50fa-45b6-9f33-88580d0cafdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:20.361130 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ca628-50fa-45b6-9f33-88580d0cafdb admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:20.365945 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadcd4bb-c261-46e4-9c09-853b23b895c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:20.366692 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadcd4bb-c261-46e4-9c09-853b23b895c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:20.366976 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadcd4bb-c261-46e4-9c09-853b23b895c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:20.367247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadcd4bb-c261-46e4-9c09-853b23b895c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:20.377802 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadcd4bb-c261-46e4-9c09-853b23b895c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:20.379448 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadcd4bb-c261-46e4-9c09-853b23b895c5 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:20.390693 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1249/2500] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:21.238160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab3bf5-c10e-4a77-942a-2aeedc3b2f87 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:21.240991 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2fab3bf5-c10e-4a77-942a-2aeedc3b2f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:21.256478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab3bf5-c10e-4a77-942a-2aeedc3b2f87 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:21.266549 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd88e2-0642-4030-831e-f3721ffe3264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:21.267901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd88e2-0642-4030-831e-f3721ffe3264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:21.267901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd88e2-0642-4030-831e-f3721ffe3264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:21.267901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd88e2-0642-4030-831e-f3721ffe3264 admin admin] PATH_INFO: `/v3/projects/336c8692bb0649a5829ff643f5d18bb3` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:21.268998 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd88e2-0642-4030-831e-f3721ffe3264 admin admin] RBAC: Authorizing `identity:delete_project(project_id=336c8692bb0649a5829ff643f5d18bb3)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:21.271146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd88e2-0642-4030-831e-f3721ffe3264 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:21.358613 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:21.364559 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e5fd88e2-0642-4030-831e-f3721ffe3264 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '336c8692bb0649a5829ff643f5d18bb3', 'request_id': 'req-e5fd88e2-0642-4030-831e-f3721ffe3264'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:21.375971 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e5fd88e2-0642-4030-831e-f3721ffe3264 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '336c8692bb0649a5829ff643f5d18bb3', 'request_id': 'req-e5fd88e2-0642-4030-831e-f3721ffe3264'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:21.384214 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e5fd88e2-0642-4030-831e-f3721ffe3264 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '336c8692bb0649a5829ff643f5d18bb3', 'request_id': 'req-e5fd88e2-0642-4030-831e-f3721ffe3264'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:21.400274 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e5fd88e2-0642-4030-831e-f3721ffe3264 admin admin] The token cache is being invalidate because project 336c8692bb0649a5829ff643f5d18bb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:21.400686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e5fd88e2-0642-4030-831e-f3721ffe3264 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:21.401549 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e5fd88e2-0642-4030-831e-f3721ffe3264 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:21.406914 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1252/2501] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:21 2022] DELETE /identity/v3/projects/336c8692bb0649a5829ff643f5d18bb3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:21.407882 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadcd4bb-c261-46e4-9c09-853b23b895c5 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:21.418563 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadcd4bb-c261-46e4-9c09-853b23b895c5 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:21.505839 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b01f26-3736-49d7-bf2a-8e98b9352918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:21.506740 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b01f26-3736-49d7-bf2a-8e98b9352918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:21.507974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b01f26-3736-49d7-bf2a-8e98b9352918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:21.508363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b01f26-3736-49d7-bf2a-8e98b9352918 admin admin] PATH_INFO: `/v3/users/6a6c01904d33403f9b8a032ee749f9dc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:21.509734 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b01f26-3736-49d7-bf2a-8e98b9352918 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a6c01904d33403f9b8a032ee749f9dc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:21.511789 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b01f26-3736-49d7-bf2a-8e98b9352918 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:21.529486 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:21.576843 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-a1b01f26-3736-49d7-bf2a-8e98b9352918 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a6c01904d33403f9b8a032ee749f9dc'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:21.577833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-a1b01f26-3736-49d7-bf2a-8e98b9352918 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a6c01904d33403f9b8a032ee749f9dc'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:21.585696 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-a1b01f26-3736-49d7-bf2a-8e98b9352918 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a6c01904d33403f9b8a032ee749f9dc'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:21.586466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-a1b01f26-3736-49d7-bf2a-8e98b9352918 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a6c01904d33403f9b8a032ee749f9dc'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:21.596731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-a1b01f26-3736-49d7-bf2a-8e98b9352918 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a6c01904d33403f9b8a032ee749f9dc'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:21.618331 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1250/2502] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:21 2022] DELETE /identity/v3/users/6a6c01904d33403f9b8a032ee749f9dc => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:21.917010 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd88e2-0642-4030-831e-f3721ffe3264 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:21.919143 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e5fd88e2-0642-4030-831e-f3721ffe3264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:21.938181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd88e2-0642-4030-831e-f3721ffe3264 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:21.997799 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2fd576-7c15-47fa-8504-70daa70722b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:21.998451 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2fd576-7c15-47fa-8504-70daa70722b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:21.998718 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2fd576-7c15-47fa-8504-70daa70722b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:21.999100 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2fd576-7c15-47fa-8504-70daa70722b2 admin admin] PATH_INFO: `/v3/projects/c5f46d9a4cc74c96a33743381f38b69d` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:22.000244 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2fd576-7c15-47fa-8504-70daa70722b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5f46d9a4cc74c96a33743381f38b69d)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:22.002171 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2fd576-7c15-47fa-8504-70daa70722b2 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:22.052302 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1e2fd576-7c15-47fa-8504-70daa70722b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5f46d9a4cc74c96a33743381f38b69d', 'request_id': 'req-1e2fd576-7c15-47fa-8504-70daa70722b2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:22.061181 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1e2fd576-7c15-47fa-8504-70daa70722b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5f46d9a4cc74c96a33743381f38b69d', 'request_id': 'req-1e2fd576-7c15-47fa-8504-70daa70722b2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:22.068320 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1e2fd576-7c15-47fa-8504-70daa70722b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5f46d9a4cc74c96a33743381f38b69d', 'request_id': 'req-1e2fd576-7c15-47fa-8504-70daa70722b2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:22.080456 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1e2fd576-7c15-47fa-8504-70daa70722b2 admin admin] The token cache is being invalidate because project c5f46d9a4cc74c96a33743381f38b69d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:22.080885 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1e2fd576-7c15-47fa-8504-70daa70722b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:22.081558 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-1e2fd576-7c15-47fa-8504-70daa70722b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:22.083061 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1253/2503] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:21 2022] DELETE /identity/v3/projects/c5f46d9a4cc74c96a33743381f38b69d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:22.089581 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f59cb-0a99-4580-9fea-9312493b1bc9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:22.089868 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f59cb-0a99-4580-9fea-9312493b1bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:22.090145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f59cb-0a99-4580-9fea-9312493b1bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:22.112230 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-b07f59cb-0a99-4580-9fea-9312493b1bc9 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:22.113372 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b07f59cb-0a99-4580-9fea-9312493b1bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:22.180415 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1251/2504] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:22.190340 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09cb5a-f825-491b-bc46-5369fa5781d4 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:22.190340 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09cb5a-f825-491b-bc46-5369fa5781d4 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:22.190567 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09cb5a-f825-491b-bc46-5369fa5781d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:22.236836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-0a09cb5a-f825-491b-bc46-5369fa5781d4 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:22.238768 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0a09cb5a-f825-491b-bc46-5369fa5781d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:22.249871 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1254/2505] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:22.254207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f59cb-0a99-4580-9fea-9312493b1bc9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:22.274371 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f59cb-0a99-4580-9fea-9312493b1bc9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:22.289617 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977d3c5c-aba2-4913-8467-a466f433ddf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:22.290244 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d3c5c-aba2-4913-8467-a466f433ddf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:22.290523 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d3c5c-aba2-4913-8467-a466f433ddf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:22.290796 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977d3c5c-aba2-4913-8467-a466f433ddf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:22.291461 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977d3c5c-aba2-4913-8467-a466f433ddf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:16:22.291794 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977d3c5c-aba2-4913-8467-a466f433ddf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:22.293885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977d3c5c-aba2-4913-8467-a466f433ddf6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:22.309949 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1252/2506] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:16:22.315658 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a09cb5a-f825-491b-bc46-5369fa5781d4 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:22.335433 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a09cb5a-f825-491b-bc46-5369fa5781d4 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:22.340438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758745a6-b9e9-4aaf-a76f-4f066616325c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:22.341053 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758745a6-b9e9-4aaf-a76f-4f066616325c admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:22.341963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758745a6-b9e9-4aaf-a76f-4f066616325c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:22.342254 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758745a6-b9e9-4aaf-a76f-4f066616325c admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:22.342989 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758745a6-b9e9-4aaf-a76f-4f066616325c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:22.345164 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758745a6-b9e9-4aaf-a76f-4f066616325c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:22.380594 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1255/2507] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:22 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:22.385066 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977d3c5c-aba2-4913-8467-a466f433ddf6 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:22.401493 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977d3c5c-aba2-4913-8467-a466f433ddf6 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:22.480413 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d5d92-67ee-4844-9af7-78e102d1c4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:22.481753 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d5d92-67ee-4844-9af7-78e102d1c4fe admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:22.482040 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d5d92-67ee-4844-9af7-78e102d1c4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:22.482357 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d5d92-67ee-4844-9af7-78e102d1c4fe admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:22.483040 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d5d92-67ee-4844-9af7-78e102d1c4fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:22.486200 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d5d92-67ee-4844-9af7-78e102d1c4fe admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:22.509884 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1253/2508] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:22 2022] POST /identity/v3/users => generated 288 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:22.514084 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758745a6-b9e9-4aaf-a76f-4f066616325c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:22.525436 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758745a6-b9e9-4aaf-a76f-4f066616325c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:22.530554 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18102970-2bb6-41cd-a134-b2ab8909690f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:22.531085 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18102970-2bb6-41cd-a134-b2ab8909690f admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:22.531370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18102970-2bb6-41cd-a134-b2ab8909690f admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:22.531759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18102970-2bb6-41cd-a134-b2ab8909690f admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:22.532404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18102970-2bb6-41cd-a134-b2ab8909690f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:22.534012 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18102970-2bb6-41cd-a134-b2ab8909690f admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:22.543988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1256/2509] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:16:22.548730 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d5d92-67ee-4844-9af7-78e102d1c4fe admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:22.559742 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d5d92-67ee-4844-9af7-78e102d1c4fe admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:22.565581 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca03affc-378b-430e-ae81-9ee3d2be775d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:22.566411 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03affc-378b-430e-ae81-9ee3d2be775d admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:22.566695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03affc-378b-430e-ae81-9ee3d2be775d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:22.566966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca03affc-378b-430e-ae81-9ee3d2be775d admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:22.567648 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03affc-378b-430e-ae81-9ee3d2be775d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:22.569252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca03affc-378b-430e-ae81-9ee3d2be775d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:22.575024 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:22.584047 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1254/2510] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:22.585576 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18102970-2bb6-41cd-a134-b2ab8909690f admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:22.600116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18102970-2bb6-41cd-a134-b2ab8909690f admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:22.612322 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479a465-d251-472a-b617-f9adb45be0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:22.613354 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479a465-d251-472a-b617-f9adb45be0e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:22.613513 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479a465-d251-472a-b617-f9adb45be0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:22.613976 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479a465-d251-472a-b617-f9adb45be0e6 admin admin] PATH_INFO: `/v3/projects/14bff8cfa2944c89b6160b3bd598e34a/users/1c8ee3a05247456aa26c08eb9ae6b43d/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:22.633208 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479a465-d251-472a-b617-f9adb45be0e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14bff8cfa2944c89b6160b3bd598e34a, user_id=1c8ee3a05247456aa26c08eb9ae6b43d, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:22.636382 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479a465-d251-472a-b617-f9adb45be0e6 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:22.648777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:22.656474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1257/2511] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:22 2022] PUT /identity/v3/projects/14bff8cfa2944c89b6160b3bd598e34a/users/1c8ee3a05247456aa26c08eb9ae6b43d/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:22.667080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf2e23-d484-4681-be83-a55303e77cf4 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:22.667744 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf2e23-d484-4681-be83-a55303e77cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:22.668148 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf2e23-d484-4681-be83-a55303e77cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:22.708893 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0fbf2e23-d484-4681-be83-a55303e77cf4 None None] MFA Rules not processed for user `1c8ee3a05247456aa26c08eb9ae6b43d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:22.710225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0fbf2e23-d484-4681-be83-a55303e77cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:22.822931 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1255/2512] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:16:22.824217 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479a465-d251-472a-b617-f9adb45be0e6 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:22.912703 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479a465-d251-472a-b617-f9adb45be0e6 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:22.920114 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8d6b7-3b53-431d-8782-4365e810fab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:22.921338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8d6b7-3b53-431d-8782-4365e810fab6 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:22.921967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8d6b7-3b53-431d-8782-4365e810fab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:22.922413 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8d6b7-3b53-431d-8782-4365e810fab6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:22.923211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8d6b7-3b53-431d-8782-4365e810fab6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:22.925382 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8d6b7-3b53-431d-8782-4365e810fab6 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:22.957651 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1258/2513] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:22 2022] POST /identity/v3/projects => generated 375 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:22.962554 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf2e23-d484-4681-be83-a55303e77cf4 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:22.976190 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf2e23-d484-4681-be83-a55303e77cf4 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:23.052052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a38300-0cf2-4e32-a91f-5c69cef4b61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:23.052747 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a38300-0cf2-4e32-a91f-5c69cef4b61c admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:23.053035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a38300-0cf2-4e32-a91f-5c69cef4b61c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:23.053312 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a38300-0cf2-4e32-a91f-5c69cef4b61c admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:23.053960 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a38300-0cf2-4e32-a91f-5c69cef4b61c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:23.055952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a38300-0cf2-4e32-a91f-5c69cef4b61c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:23.091600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1256/2514] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:22 2022] POST /identity/v3/users => generated 294 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:16:23.095412 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8d6b7-3b53-431d-8782-4365e810fab6 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:23.106552 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8d6b7-3b53-431d-8782-4365e810fab6 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:23.111922 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e47b9-f2a4-420c-9c9b-c9da0936ec36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:23.112526 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e47b9-f2a4-420c-9c9b-c9da0936ec36 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:23.112817 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e47b9-f2a4-420c-9c9b-c9da0936ec36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:23.113106 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e47b9-f2a4-420c-9c9b-c9da0936ec36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:23.113732 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e47b9-f2a4-420c-9c9b-c9da0936ec36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:23.115341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e47b9-f2a4-420c-9c9b-c9da0936ec36 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:23.127152 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1259/2515] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:16:23.131304 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a38300-0cf2-4e32-a91f-5c69cef4b61c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:23.141741 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a38300-0cf2-4e32-a91f-5c69cef4b61c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:23.146838 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8783313a-3d13-42a3-b31e-5225f938ccc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:23.147502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783313a-3d13-42a3-b31e-5225f938ccc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:23.147814 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783313a-3d13-42a3-b31e-5225f938ccc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:23.148088 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783313a-3d13-42a3-b31e-5225f938ccc3 admin admin] PATH_INFO: `/v3/projects/07ee9a9e4c89439497de43cc49da826d/users/0d0868a9d6494a4e95c236cf9483ac5d/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:23.165195 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8783313a-3d13-42a3-b31e-5225f938ccc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07ee9a9e4c89439497de43cc49da826d, user_id=0d0868a9d6494a4e95c236cf9483ac5d, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:23.168623 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8783313a-3d13-42a3-b31e-5225f938ccc3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:23.188640 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1257/2516] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:23 2022] PUT /identity/v3/projects/07ee9a9e4c89439497de43cc49da826d/users/0d0868a9d6494a4e95c236cf9483ac5d/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:16:23.192752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e47b9-f2a4-420c-9c9b-c9da0936ec36 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:23.203077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e47b9-f2a4-420c-9c9b-c9da0936ec36 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:23.261618 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8783313a-3d13-42a3-b31e-5225f938ccc3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:23.265285 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8783313a-3d13-42a3-b31e-5225f938ccc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:23.267821 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60521640-c6f1-4ed2-baf3-45b174e983e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:23.268448 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60521640-c6f1-4ed2-baf3-45b174e983e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:23.268733 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60521640-c6f1-4ed2-baf3-45b174e983e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:23.269012 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60521640-c6f1-4ed2-baf3-45b174e983e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:23.269660 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60521640-c6f1-4ed2-baf3-45b174e983e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:23.271262 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60521640-c6f1-4ed2-baf3-45b174e983e1 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:23.281473 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8783313a-3d13-42a3-b31e-5225f938ccc3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:23.286810 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1260/2517] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:23.288030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60521640-c6f1-4ed2-baf3-45b174e983e1 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:23.290625 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6c08b-783e-4af1-913e-e628032a6052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:23.291414 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6c08b-783e-4af1-913e-e628032a6052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:23.295608 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6c08b-783e-4af1-913e-e628032a6052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:23.295608 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6c08b-783e-4af1-913e-e628032a6052 admin admin] PATH_INFO: `/v3/users/51768890f19140b6819ef89f5bf610c8` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:23.295608 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6c08b-783e-4af1-913e-e628032a6052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51768890f19140b6819ef89f5bf610c8)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:23.297333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6c08b-783e-4af1-913e-e628032a6052 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:23.300232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60521640-c6f1-4ed2-baf3-45b174e983e1 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:23.306291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33400965-a480-44ed-9ed8-bcf29d25224f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:23.307093 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33400965-a480-44ed-9ed8-bcf29d25224f admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:23.307452 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33400965-a480-44ed-9ed8-bcf29d25224f admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:23.307829 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33400965-a480-44ed-9ed8-bcf29d25224f admin admin] PATH_INFO: `/v3/projects/07ee9a9e4c89439497de43cc49da826d/users/0d0868a9d6494a4e95c236cf9483ac5d/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:23.310305 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33400965-a480-44ed-9ed8-bcf29d25224f admin admin] RBAC: Authorizing `identity:create_grant(project_id=07ee9a9e4c89439497de43cc49da826d, user_id=0d0868a9d6494a4e95c236cf9483ac5d, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:23.313853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33400965-a480-44ed-9ed8-bcf29d25224f admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:23.329539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1261/2518] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:23 2022] PUT /identity/v3/projects/07ee9a9e4c89439497de43cc49da826d/users/0d0868a9d6494a4e95c236cf9483ac5d/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:16:23.342508 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a19a5d-4421-47cb-b444-028750c0c55e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:23.350600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a19a5d-4421-47cb-b444-028750c0c55e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:23.351002 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a19a5d-4421-47cb-b444-028750c0c55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:23.363727 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dfa6c08b-783e-4af1-913e-e628032a6052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51768890f19140b6819ef89f5bf610c8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:23.364481 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dfa6c08b-783e-4af1-913e-e628032a6052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51768890f19140b6819ef89f5bf610c8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:23.367376 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:23.376947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dfa6c08b-783e-4af1-913e-e628032a6052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51768890f19140b6819ef89f5bf610c8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:23.379637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dfa6c08b-783e-4af1-913e-e628032a6052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51768890f19140b6819ef89f5bf610c8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:23.384801 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e0a19a5d-4421-47cb-b444-028750c0c55e None None] MFA Rules not processed for user `0d0868a9d6494a4e95c236cf9483ac5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:23.386031 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e0a19a5d-4421-47cb-b444-028750c0c55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:23.388404 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dfa6c08b-783e-4af1-913e-e628032a6052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51768890f19140b6819ef89f5bf610c8'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:23.409696 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1258/2519] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:23 2022] DELETE /identity/v3/users/51768890f19140b6819ef89f5bf610c8 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:16:23.437005 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:23.463229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1262/2520] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:23.475196 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6c08b-783e-4af1-913e-e628032a6052 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:23.477682 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dfa6c08b-783e-4af1-913e-e628032a6052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:23.551174 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6c08b-783e-4af1-913e-e628032a6052 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:23.556934 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d7500-adee-442c-8ec2-8e719ed20dae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:23.558004 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d7500-adee-442c-8ec2-8e719ed20dae service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:23.558301 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d7500-adee-442c-8ec2-8e719ed20dae service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:23.558755 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d7500-adee-442c-8ec2-8e719ed20dae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:23.560592 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8c7d7500-adee-442c-8ec2-8e719ed20dae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:23.577399 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a19a5d-4421-47cb-b444-028750c0c55e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:23.578866 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e0a19a5d-4421-47cb-b444-028750c0c55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:23.636423 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d7500-adee-442c-8ec2-8e719ed20dae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:23.638228 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d7500-adee-442c-8ec2-8e719ed20dae service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:23.648396 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1259/2521] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:23.650181 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d7500-adee-442c-8ec2-8e719ed20dae service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:23.651473 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8c7d7500-adee-442c-8ec2-8e719ed20dae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:23.680763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a19a5d-4421-47cb-b444-028750c0c55e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:23.729323 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d7500-adee-442c-8ec2-8e719ed20dae service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:23.735025 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffed240-e6ff-41f1-b516-87b7a1bca732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:23.735772 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffed240-e6ff-41f1-b516-87b7a1bca732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:23.735981 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffed240-e6ff-41f1-b516-87b7a1bca732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:23.736242 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffed240-e6ff-41f1-b516-87b7a1bca732 admin admin] PATH_INFO: `/v3/projects/62cb41eb4b8244abb4aafd718b1207ad` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:23.737372 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffed240-e6ff-41f1-b516-87b7a1bca732 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62cb41eb4b8244abb4aafd718b1207ad)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:23.738410 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c7ecd-e3b3-4cc0-b47d-9be36387c208 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:23.739288 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffed240-e6ff-41f1-b516-87b7a1bca732 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:23.741171 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c7ecd-e3b3-4cc0-b47d-9be36387c208 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:23.741459 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c7ecd-e3b3-4cc0-b47d-9be36387c208 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:23.741735 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c7ecd-e3b3-4cc0-b47d-9be36387c208 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:23.743988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f55c7ecd-e3b3-4cc0-b47d-9be36387c208 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:23.759374 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c7ecd-e3b3-4cc0-b47d-9be36387c208 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:23.761087 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c7ecd-e3b3-4cc0-b47d-9be36387c208 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:23.766899 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=4,keystone:SELECT=96 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:23.772813 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1263/2522] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:16:23.795059 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fffed240-e6ff-41f1-b516-87b7a1bca732 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62cb41eb4b8244abb4aafd718b1207ad', 'request_id': 'req-fffed240-e6ff-41f1-b516-87b7a1bca732'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:23.805662 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fffed240-e6ff-41f1-b516-87b7a1bca732 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62cb41eb4b8244abb4aafd718b1207ad', 'request_id': 'req-fffed240-e6ff-41f1-b516-87b7a1bca732'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:23.812646 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fffed240-e6ff-41f1-b516-87b7a1bca732 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62cb41eb4b8244abb4aafd718b1207ad', 'request_id': 'req-fffed240-e6ff-41f1-b516-87b7a1bca732'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:23.832305 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fffed240-e6ff-41f1-b516-87b7a1bca732 admin admin] The token cache is being invalidate because project 62cb41eb4b8244abb4aafd718b1207ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:23.832797 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fffed240-e6ff-41f1-b516-87b7a1bca732 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:23.833652 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fffed240-e6ff-41f1-b516-87b7a1bca732 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:23.835767 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1260/2523] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:23 2022] DELETE /identity/v3/projects/62cb41eb4b8244abb4aafd718b1207ad => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:23.840965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71beefd-5ef5-419e-bf6d-06a4c84a69ed None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:23.841446 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71beefd-5ef5-419e-bf6d-06a4c84a69ed None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:23.841872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71beefd-5ef5-419e-bf6d-06a4c84a69ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:23.867285 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a71beefd-5ef5-419e-bf6d-06a4c84a69ed None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:23.869873 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a71beefd-5ef5-419e-bf6d-06a4c84a69ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:23.976256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1264/2524] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:16:23.984954 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d096e-dbcd-4699-9840-1d3b7e35650b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:23.985375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d096e-dbcd-4699-9840-1d3b7e35650b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:23.985772 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d096e-dbcd-4699-9840-1d3b7e35650b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:24.009210 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a97d096e-dbcd-4699-9840-1d3b7e35650b None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:24.010545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a97d096e-dbcd-4699-9840-1d3b7e35650b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:24.020186 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1261/2525] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:24.024268 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71beefd-5ef5-419e-bf6d-06a4c84a69ed None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:24.036676 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71beefd-5ef5-419e-bf6d-06a4c84a69ed None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:24.044019 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc697ed1-2c46-4744-ae84-d09d2f762136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:24.044652 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc697ed1-2c46-4744-ae84-d09d2f762136 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:24.045294 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc697ed1-2c46-4744-ae84-d09d2f762136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:24.045294 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc697ed1-2c46-4744-ae84-d09d2f762136 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:24.046557 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc697ed1-2c46-4744-ae84-d09d2f762136 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:16:24.046800 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc697ed1-2c46-4744-ae84-d09d2f762136 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:24.048688 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc697ed1-2c46-4744-ae84-d09d2f762136 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:24.063514 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1265/2526] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:16:24.068345 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d096e-dbcd-4699-9840-1d3b7e35650b None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:24.079779 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d096e-dbcd-4699-9840-1d3b7e35650b None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:24.084867 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e0553-2c02-4861-8a00-b2dafc58cea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:24.085480 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e0553-2c02-4861-8a00-b2dafc58cea9 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:24.085789 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e0553-2c02-4861-8a00-b2dafc58cea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:24.086062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e0553-2c02-4861-8a00-b2dafc58cea9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:24.086711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e0553-2c02-4861-8a00-b2dafc58cea9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:24.088657 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e0553-2c02-4861-8a00-b2dafc58cea9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:24.117358 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1262/2527] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:24 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:16:24.121707 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc697ed1-2c46-4744-ae84-d09d2f762136 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:24.133313 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc697ed1-2c46-4744-ae84-d09d2f762136 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:24.161120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:24.195090 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffc294-bb5a-47c3-af16-b15c57dda337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:24.195751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffc294-bb5a-47c3-af16-b15c57dda337 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:24.196036 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffc294-bb5a-47c3-af16-b15c57dda337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:24.196313 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffc294-bb5a-47c3-af16-b15c57dda337 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:24.196958 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbffc294-bb5a-47c3-af16-b15c57dda337 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:24.198892 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbffc294-bb5a-47c3-af16-b15c57dda337 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:24.222322 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1266/2528] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:24 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:24.225447 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e0553-2c02-4861-8a00-b2dafc58cea9 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:24.236032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e0553-2c02-4861-8a00-b2dafc58cea9 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:24.241411 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99090bab-a1df-4923-868d-6753bbf6f546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:24.242060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99090bab-a1df-4923-868d-6753bbf6f546 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:24.242390 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99090bab-a1df-4923-868d-6753bbf6f546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:24.242759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99090bab-a1df-4923-868d-6753bbf6f546 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:24.243445 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99090bab-a1df-4923-868d-6753bbf6f546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:24.245126 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99090bab-a1df-4923-868d-6753bbf6f546 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:24.259687 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1263/2529] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:24.260955 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffc294-bb5a-47c3-af16-b15c57dda337 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:24.272185 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffc294-bb5a-47c3-af16-b15c57dda337 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:24.276925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eee119-cdf9-478e-a057-1a20c1723bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:24.277533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eee119-cdf9-478e-a057-1a20c1723bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:24.277812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eee119-cdf9-478e-a057-1a20c1723bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:24.278085 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eee119-cdf9-478e-a057-1a20c1723bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:24.278749 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eee119-cdf9-478e-a057-1a20c1723bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:24.280398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eee119-cdf9-478e-a057-1a20c1723bb4 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:24.291579 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1267/2530] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:16:24.295682 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99090bab-a1df-4923-868d-6753bbf6f546 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:24.305988 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99090bab-a1df-4923-868d-6753bbf6f546 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:24.310775 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba65ff-7ec5-408c-b97b-2bf695cb0681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:24.312110 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba65ff-7ec5-408c-b97b-2bf695cb0681 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:24.312444 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba65ff-7ec5-408c-b97b-2bf695cb0681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:24.312715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba65ff-7ec5-408c-b97b-2bf695cb0681 admin admin] PATH_INFO: `/v3/projects/d7e277b0055d4b1d8e1c48a60347b38b/users/8d9a9e979214488dab6a541a72ea269a/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:24.328333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba65ff-7ec5-408c-b97b-2bf695cb0681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7e277b0055d4b1d8e1c48a60347b38b, user_id=8d9a9e979214488dab6a541a72ea269a, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:24.331206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba65ff-7ec5-408c-b97b-2bf695cb0681 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:24.344272 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1264/2531] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:24 2022] PUT /identity/v3/projects/d7e277b0055d4b1d8e1c48a60347b38b/users/8d9a9e979214488dab6a541a72ea269a/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:24.354334 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce3fbd-8436-42ea-a6e3-06b29d2f2df2 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:24.354625 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce3fbd-8436-42ea-a6e3-06b29d2f2df2 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:24.354902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce3fbd-8436-42ea-a6e3-06b29d2f2df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:24.386037 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e7ce3fbd-8436-42ea-a6e3-06b29d2f2df2 None None] MFA Rules not processed for user `8d9a9e979214488dab6a541a72ea269a`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:24.387338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e7ce3fbd-8436-42ea-a6e3-06b29d2f2df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:24.455578 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1268/2532] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:24.460165 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba65ff-7ec5-408c-b97b-2bf695cb0681 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:24.528282 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba65ff-7ec5-408c-b97b-2bf695cb0681 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:24.532972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4955d1-8720-45c2-b06a-65cf589d9015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:24.533618 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4955d1-8720-45c2-b06a-65cf589d9015 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:24.533905 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4955d1-8720-45c2-b06a-65cf589d9015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:24.534184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4955d1-8720-45c2-b06a-65cf589d9015 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:24.534833 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4955d1-8720-45c2-b06a-65cf589d9015 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:24.536800 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4955d1-8720-45c2-b06a-65cf589d9015 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:24.567261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1265/2533] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:24 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:16:24.569185 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce3fbd-8436-42ea-a6e3-06b29d2f2df2 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:24.579621 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce3fbd-8436-42ea-a6e3-06b29d2f2df2 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:24.638232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a9468-fe27-4b76-b75f-9a36532300a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:24.638841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a9468-fe27-4b76-b75f-9a36532300a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:24.639124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a9468-fe27-4b76-b75f-9a36532300a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:24.639398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a9468-fe27-4b76-b75f-9a36532300a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:24.640304 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a9468-fe27-4b76-b75f-9a36532300a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:24.642377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a9468-fe27-4b76-b75f-9a36532300a4 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:24.663766 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1269/2534] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:24 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:24.667507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4955d1-8720-45c2-b06a-65cf589d9015 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:24.679025 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4955d1-8720-45c2-b06a-65cf589d9015 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:24.686377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b1ca1-ae05-4e0a-8454-b39b9b104d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:24.687846 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b1ca1-ae05-4e0a-8454-b39b9b104d46 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:24.688250 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b1ca1-ae05-4e0a-8454-b39b9b104d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:24.688529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b1ca1-ae05-4e0a-8454-b39b9b104d46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:24.689179 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0b1ca1-ae05-4e0a-8454-b39b9b104d46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:24.694197 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0b1ca1-ae05-4e0a-8454-b39b9b104d46 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:24.706598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1266/2535] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:24.710755 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a9468-fe27-4b76-b75f-9a36532300a4 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:24.723124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a9468-fe27-4b76-b75f-9a36532300a4 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:24.729813 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1e256-58c9-4085-896c-420363176acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:24.730428 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1e256-58c9-4085-896c-420363176acb admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:24.730713 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1e256-58c9-4085-896c-420363176acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:24.730988 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1e256-58c9-4085-896c-420363176acb admin admin] PATH_INFO: `/v3/projects/28e918f14a6b47338303abb0d3670926/users/0a3ae2fb249c4c9ab17e07928974cf09/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:24.748003 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1e256-58c9-4085-896c-420363176acb admin admin] RBAC: Authorizing `identity:create_grant(project_id=28e918f14a6b47338303abb0d3670926, user_id=0a3ae2fb249c4c9ab17e07928974cf09, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:24.751781 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1e256-58c9-4085-896c-420363176acb admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:24.767434 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1270/2536] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:24 2022] PUT /identity/v3/projects/28e918f14a6b47338303abb0d3670926/users/0a3ae2fb249c4c9ab17e07928974cf09/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:16:24.771311 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b1ca1-ae05-4e0a-8454-b39b9b104d46 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:24.783493 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b1ca1-ae05-4e0a-8454-b39b9b104d46 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:24.805365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:24.861082 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a86233-ad9d-4f8f-b489-5e633d76e0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:24.861801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a86233-ad9d-4f8f-b489-5e633d76e0ba admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:24.862088 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a86233-ad9d-4f8f-b489-5e633d76e0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:24.862364 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a86233-ad9d-4f8f-b489-5e633d76e0ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:24.863014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a86233-ad9d-4f8f-b489-5e633d76e0ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:24.865204 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a86233-ad9d-4f8f-b489-5e633d76e0ba admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:24.876675 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1267/2537] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:24.881081 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1e256-58c9-4085-896c-420363176acb admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:24.886739 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:24.894307 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1e256-58c9-4085-896c-420363176acb admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:24.899350 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c54933-4ab9-46fb-92f5-f17700d4e1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:24.900118 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c54933-4ab9-46fb-92f5-f17700d4e1a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:24.900496 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c54933-4ab9-46fb-92f5-f17700d4e1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:24.900881 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c54933-4ab9-46fb-92f5-f17700d4e1a3 admin admin] PATH_INFO: `/v3/projects/28e918f14a6b47338303abb0d3670926/users/0a3ae2fb249c4c9ab17e07928974cf09/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:24.903108 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c54933-4ab9-46fb-92f5-f17700d4e1a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28e918f14a6b47338303abb0d3670926, user_id=0a3ae2fb249c4c9ab17e07928974cf09, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:24.906263 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c54933-4ab9-46fb-92f5-f17700d4e1a3 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:24.917184 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1271/2538] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:24 2022] PUT /identity/v3/projects/28e918f14a6b47338303abb0d3670926/users/0a3ae2fb249c4c9ab17e07928974cf09/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:24.928889 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c4b39-5afe-4892-aa6b-65bf69aede24 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:24.929180 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c4b39-5afe-4892-aa6b-65bf69aede24 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:24.929457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c4b39-5afe-4892-aa6b-65bf69aede24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:24.961795 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-6f2c4b39-5afe-4892-aa6b-65bf69aede24 None None] MFA Rules not processed for user `0a3ae2fb249c4c9ab17e07928974cf09`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:24.962927 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6f2c4b39-5afe-4892-aa6b-65bf69aede24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:25.032426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1268/2539] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:24 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:25.044941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c54933-4ab9-46fb-92f5-f17700d4e1a3 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:25.047065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-83c54933-4ab9-46fb-92f5-f17700d4e1a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:25.119246 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c54933-4ab9-46fb-92f5-f17700d4e1a3 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:25.125409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1effeb8e-c9c6-406a-b6c4-2d6c5d352229 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:25.126349 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effeb8e-c9c6-406a-b6c4-2d6c5d352229 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:25.127135 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effeb8e-c9c6-406a-b6c4-2d6c5d352229 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:25.127412 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effeb8e-c9c6-406a-b6c4-2d6c5d352229 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:25.138090 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1effeb8e-c9c6-406a-b6c4-2d6c5d352229 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:25.140220 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1effeb8e-c9c6-406a-b6c4-2d6c5d352229 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:25.152479 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1272/2540] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:25.575456 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c4b39-5afe-4892-aa6b-65bf69aede24 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:25.585748 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c4b39-5afe-4892-aa6b-65bf69aede24 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:25.590816 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e9430-55a6-4497-9995-f5e81dbda79f tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ae2fb249c4c9ab17e07928974cf09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28e918f14a6b47338303abb0d3670926', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:25.591437 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e9430-55a6-4497-9995-f5e81dbda79f tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:25.591785 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e9430-55a6-4497-9995-f5e81dbda79f tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:25.592145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e9430-55a6-4497-9995-f5e81dbda79f tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:25.593394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e9430-55a6-4497-9995-f5e81dbda79f tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:16:25.594058 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e9430-55a6-4497-9995-f5e81dbda79f tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:25.595984 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e9430-55a6-4497-9995-f5e81dbda79f tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:25.608338 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1269/2541] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:16:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:16:25.612359 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1effeb8e-c9c6-406a-b6c4-2d6c5d352229 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:25.622193 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1effeb8e-c9c6-406a-b6c4-2d6c5d352229 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:25.628468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50870061-536a-41fb-a834-077fd5aed26a tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ae2fb249c4c9ab17e07928974cf09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28e918f14a6b47338303abb0d3670926', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:25.629116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50870061-536a-41fb-a834-077fd5aed26a tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:25.629538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50870061-536a-41fb-a834-077fd5aed26a tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:25.629816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50870061-536a-41fb-a834-077fd5aed26a tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:25.630943 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50870061-536a-41fb-a834-077fd5aed26a tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:25.632510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50870061-536a-41fb-a834-077fd5aed26a tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:25.663589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1273/2542] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:25 2022] POST /identity/v3/projects => generated 355 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:16:25.667305 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e9430-55a6-4497-9995-f5e81dbda79f tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:25.678443 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e9430-55a6-4497-9995-f5e81dbda79f tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:25.746588 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d1541-39a7-4383-9ec1-19da29fa6c9c tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ae2fb249c4c9ab17e07928974cf09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28e918f14a6b47338303abb0d3670926', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:25.747520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d1541-39a7-4383-9ec1-19da29fa6c9c tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:25.747674 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d1541-39a7-4383-9ec1-19da29fa6c9c tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:25.748152 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d1541-39a7-4383-9ec1-19da29fa6c9c tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:25.749566 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d1541-39a7-4383-9ec1-19da29fa6c9c tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:16:25.749872 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d1541-39a7-4383-9ec1-19da29fa6c9c tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:25.751939 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d1541-39a7-4383-9ec1-19da29fa6c9c tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:25.766382 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1270/2543] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:16:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:16:25.807882 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50870061-536a-41fb-a834-077fd5aed26a tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:25.825397 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50870061-536a-41fb-a834-077fd5aed26a tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:25.830645 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48b2b5-b296-49ca-9df3-4c1baca28755 tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a3ae2fb249c4c9ab17e07928974cf09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28e918f14a6b47338303abb0d3670926', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:25.831294 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48b2b5-b296-49ca-9df3-4c1baca28755 tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:25.831596 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48b2b5-b296-49ca-9df3-4c1baca28755 tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:25.831875 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48b2b5-b296-49ca-9df3-4c1baca28755 tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] PATH_INFO: `/v3/projects/6514ded78b6745d48760461669b84d3c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:25.833260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae48b2b5-b296-49ca-9df3-4c1baca28755 tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6514ded78b6745d48760461669b84d3c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:25.835381 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae48b2b5-b296-49ca-9df3-4c1baca28755 tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:25.895657 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ae48b2b5-b296-49ca-9df3-4c1baca28755 tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6514ded78b6745d48760461669b84d3c', 'request_id': 'req-ae48b2b5-b296-49ca-9df3-4c1baca28755'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:25.903304 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ae48b2b5-b296-49ca-9df3-4c1baca28755 tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6514ded78b6745d48760461669b84d3c', 'request_id': 'req-ae48b2b5-b296-49ca-9df3-4c1baca28755'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:25.909568 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ae48b2b5-b296-49ca-9df3-4c1baca28755 tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6514ded78b6745d48760461669b84d3c', 'request_id': 'req-ae48b2b5-b296-49ca-9df3-4c1baca28755'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:25.921665 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ae48b2b5-b296-49ca-9df3-4c1baca28755 tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] The token cache is being invalidate because project 6514ded78b6745d48760461669b84d3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:25.921865 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ae48b2b5-b296-49ca-9df3-4c1baca28755 tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:25.922569 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ae48b2b5-b296-49ca-9df3-4c1baca28755 tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:25.924150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1274/2544] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:25 2022] DELETE /identity/v3/projects/6514ded78b6745d48760461669b84d3c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:26.132410 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d1541-39a7-4383-9ec1-19da29fa6c9c tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:26.134529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a29d1541-39a7-4383-9ec1-19da29fa6c9c tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:26.168680 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:26.215473 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d1541-39a7-4383-9ec1-19da29fa6c9c tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:26.281582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03058886-3a00-4b02-b956-80592a5d2528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:26.282229 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03058886-3a00-4b02-b956-80592a5d2528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:26.282509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03058886-3a00-4b02-b956-80592a5d2528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:26.282781 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03058886-3a00-4b02-b956-80592a5d2528 admin admin] PATH_INFO: `/v3/users/8d9a9e979214488dab6a541a72ea269a` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:26.283968 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03058886-3a00-4b02-b956-80592a5d2528 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d9a9e979214488dab6a541a72ea269a)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:26.285929 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03058886-3a00-4b02-b956-80592a5d2528 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:26.292749 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48b2b5-b296-49ca-9df3-4c1baca28755 tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:26.294166 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ae48b2b5-b296-49ca-9df3-4c1baca28755 tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:26.309351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48b2b5-b296-49ca-9df3-4c1baca28755 tempest-VolumeQuotaClassesTest-681276974 tempest-VolumeQuotaClassesTest-681276974-project-admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:26.314842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c5d93-087b-41ea-95e0-5124c774f368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:26.315485 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c5d93-087b-41ea-95e0-5124c774f368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:26.315819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c5d93-087b-41ea-95e0-5124c774f368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:26.316098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c5d93-087b-41ea-95e0-5124c774f368 admin admin] PATH_INFO: `/v3/users/1c8ee3a05247456aa26c08eb9ae6b43d` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:26.317231 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c5d93-087b-41ea-95e0-5124c774f368 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c8ee3a05247456aa26c08eb9ae6b43d)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:26.319178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c5d93-087b-41ea-95e0-5124c774f368 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:26.377815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-03058886-3a00-4b02-b956-80592a5d2528 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d9a9e979214488dab6a541a72ea269a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:26.378569 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-03058886-3a00-4b02-b956-80592a5d2528 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d9a9e979214488dab6a541a72ea269a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:26.389037 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-03058886-3a00-4b02-b956-80592a5d2528 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d9a9e979214488dab6a541a72ea269a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:26.389678 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-03058886-3a00-4b02-b956-80592a5d2528 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d9a9e979214488dab6a541a72ea269a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:26.393705 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ad7c5d93-087b-41ea-95e0-5124c774f368 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c8ee3a05247456aa26c08eb9ae6b43d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:26.394860 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ad7c5d93-087b-41ea-95e0-5124c774f368 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c8ee3a05247456aa26c08eb9ae6b43d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:26.398468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=83,keystone:INSERT=3,keystone:DELETE=14 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:26.402071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-03058886-3a00-4b02-b956-80592a5d2528 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d9a9e979214488dab6a541a72ea269a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:26.412302 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ad7c5d93-087b-41ea-95e0-5124c774f368 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c8ee3a05247456aa26c08eb9ae6b43d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:26.427836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ad7c5d93-087b-41ea-95e0-5124c774f368 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c8ee3a05247456aa26c08eb9ae6b43d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:26.428795 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ad7c5d93-087b-41ea-95e0-5124c774f368 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c8ee3a05247456aa26c08eb9ae6b43d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:26.435616 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1271/2545] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:26 2022] DELETE /identity/v3/users/8d9a9e979214488dab6a541a72ea269a => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:26.440694 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e4c7e-4025-4a82-b5f7-e6845ce3e2b8 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:26.440977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e4c7e-4025-4a82-b5f7-e6845ce3e2b8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:26.441259 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e4c7e-4025-4a82-b5f7-e6845ce3e2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:26.456588 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1275/2546] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:26 2022] DELETE /identity/v3/users/1c8ee3a05247456aa26c08eb9ae6b43d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:26.460569 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17403b-66bc-4364-b124-425b21f4a9bf None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:26.460968 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17403b-66bc-4364-b124-425b21f4a9bf None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:26.461345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17403b-66bc-4364-b124-425b21f4a9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:26.481745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-222e4c7e-4025-4a82-b5f7-e6845ce3e2b8 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:26.483336 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-222e4c7e-4025-4a82-b5f7-e6845ce3e2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:26.501143 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-ca17403b-66bc-4364-b124-425b21f4a9bf None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:26.502950 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ca17403b-66bc-4364-b124-425b21f4a9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:26.557863 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:26.592583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1272/2547] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:16:26.595531 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1276/2548] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:16:26.609044 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17403b-66bc-4364-b124-425b21f4a9bf None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:26.609270 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e4c7e-4025-4a82-b5f7-e6845ce3e2b8 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:26.611428 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-222e4c7e-4025-4a82-b5f7-e6845ce3e2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:26.611696 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ca17403b-66bc-4364-b124-425b21f4a9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:26.687531 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17403b-66bc-4364-b124-425b21f4a9bf None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:26.692242 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e4c7e-4025-4a82-b5f7-e6845ce3e2b8 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:26.694300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4218e-195a-444d-bd2d-7c98212a2a2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:26.694943 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4218e-195a-444d-bd2d-7c98212a2a2b service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:26.695229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4218e-195a-444d-bd2d-7c98212a2a2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:26.695936 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4218e-195a-444d-bd2d-7c98212a2a2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:26.697419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4765d2d-4ec4-4481-8b79-c5a07538f904 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:26.698243 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4765d2d-4ec4-4481-8b79-c5a07538f904 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:26.698686 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4765d2d-4ec4-4481-8b79-c5a07538f904 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:26.699347 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4765d2d-4ec4-4481-8b79-c5a07538f904 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:26.707245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4218e-195a-444d-bd2d-7c98212a2a2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:26.708973 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4218e-195a-444d-bd2d-7c98212a2a2b service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:26.712659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4765d2d-4ec4-4481-8b79-c5a07538f904 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:26.714499 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4765d2d-4ec4-4481-8b79-c5a07538f904 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:26.722749 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1277/2549] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:26.730860 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1273/2550] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:27.037803 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4218e-195a-444d-bd2d-7c98212a2a2b service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:27.039368 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-71c4218e-195a-444d-bd2d-7c98212a2a2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:27.058344 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4218e-195a-444d-bd2d-7c98212a2a2b service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:27.063414 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ac646a-01d2-4a72-bf2b-d26c29c503d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:27.064173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac646a-01d2-4a72-bf2b-d26c29c503d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:27.064450 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac646a-01d2-4a72-bf2b-d26c29c503d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:27.064720 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac646a-01d2-4a72-bf2b-d26c29c503d8 admin admin] PATH_INFO: `/v3/users/e4ad2d06f62246be8eb162257cbb4f8e` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:27.065873 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ac646a-01d2-4a72-bf2b-d26c29c503d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4ad2d06f62246be8eb162257cbb4f8e)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:27.067881 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ac646a-01d2-4a72-bf2b-d26c29c503d8 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:27.131784 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a2ac646a-01d2-4a72-bf2b-d26c29c503d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4ad2d06f62246be8eb162257cbb4f8e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.132870 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a2ac646a-01d2-4a72-bf2b-d26c29c503d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4ad2d06f62246be8eb162257cbb4f8e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.142008 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a2ac646a-01d2-4a72-bf2b-d26c29c503d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4ad2d06f62246be8eb162257cbb4f8e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.159399 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a2ac646a-01d2-4a72-bf2b-d26c29c503d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4ad2d06f62246be8eb162257cbb4f8e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.166777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a2ac646a-01d2-4a72-bf2b-d26c29c503d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4ad2d06f62246be8eb162257cbb4f8e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.168603 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:27.173553 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4765d2d-4ec4-4481-8b79-c5a07538f904 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:27.175043 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e4765d2d-4ec4-4481-8b79-c5a07538f904 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:27.196078 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4765d2d-4ec4-4481-8b79-c5a07538f904 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:27.203182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1278/2551] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:27 2022] DELETE /identity/v3/users/e4ad2d06f62246be8eb162257cbb4f8e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:27.251679 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ac646a-01d2-4a72-bf2b-d26c29c503d8 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:27.253391 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a2ac646a-01d2-4a72-bf2b-d26c29c503d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:27.301517 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742e77e-edd7-429c-8dcc-a1081045c656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:27.302179 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742e77e-edd7-429c-8dcc-a1081045c656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:27.302465 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742e77e-edd7-429c-8dcc-a1081045c656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:27.302738 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742e77e-edd7-429c-8dcc-a1081045c656 admin admin] PATH_INFO: `/v3/projects/14bff8cfa2944c89b6160b3bd598e34a` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:27.303957 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742e77e-edd7-429c-8dcc-a1081045c656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14bff8cfa2944c89b6160b3bd598e34a)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:27.306474 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742e77e-edd7-429c-8dcc-a1081045c656 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:27.346136 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ac646a-01d2-4a72-bf2b-d26c29c503d8 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:27.352824 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c886d2c-c3da-4afb-a08d-6a1002d9bf68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:27.353686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c886d2c-c3da-4afb-a08d-6a1002d9bf68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:27.354067 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c886d2c-c3da-4afb-a08d-6a1002d9bf68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:27.354448 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c886d2c-c3da-4afb-a08d-6a1002d9bf68 admin admin] PATH_INFO: `/v3/projects/d7e277b0055d4b1d8e1c48a60347b38b` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:27.356089 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c886d2c-c3da-4afb-a08d-6a1002d9bf68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7e277b0055d4b1d8e1c48a60347b38b)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:27.358115 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c886d2c-c3da-4afb-a08d-6a1002d9bf68 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:27.390155 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e742e77e-edd7-429c-8dcc-a1081045c656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14bff8cfa2944c89b6160b3bd598e34a', 'request_id': 'req-e742e77e-edd7-429c-8dcc-a1081045c656'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.406905 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e742e77e-edd7-429c-8dcc-a1081045c656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14bff8cfa2944c89b6160b3bd598e34a', 'request_id': 'req-e742e77e-edd7-429c-8dcc-a1081045c656'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.427847 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e742e77e-edd7-429c-8dcc-a1081045c656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14bff8cfa2944c89b6160b3bd598e34a', 'request_id': 'req-e742e77e-edd7-429c-8dcc-a1081045c656'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.433532 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4c886d2c-c3da-4afb-a08d-6a1002d9bf68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7e277b0055d4b1d8e1c48a60347b38b', 'request_id': 'req-4c886d2c-c3da-4afb-a08d-6a1002d9bf68'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.446401 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4c886d2c-c3da-4afb-a08d-6a1002d9bf68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7e277b0055d4b1d8e1c48a60347b38b', 'request_id': 'req-4c886d2c-c3da-4afb-a08d-6a1002d9bf68'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.447916 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e742e77e-edd7-429c-8dcc-a1081045c656 admin admin] The token cache is being invalidate because project 14bff8cfa2944c89b6160b3bd598e34a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:27.448229 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e742e77e-edd7-429c-8dcc-a1081045c656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.449062 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e742e77e-edd7-429c-8dcc-a1081045c656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.454619 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1274/2552] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:27 2022] DELETE /identity/v3/projects/14bff8cfa2944c89b6160b3bd598e34a => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:27.455438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4c886d2c-c3da-4afb-a08d-6a1002d9bf68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7e277b0055d4b1d8e1c48a60347b38b', 'request_id': 'req-4c886d2c-c3da-4afb-a08d-6a1002d9bf68'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.455578 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742e77e-edd7-429c-8dcc-a1081045c656 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:27.456910 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e742e77e-edd7-429c-8dcc-a1081045c656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:27.469719 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4c886d2c-c3da-4afb-a08d-6a1002d9bf68 admin admin] The token cache is being invalidate because project d7e277b0055d4b1d8e1c48a60347b38b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:27.470597 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4c886d2c-c3da-4afb-a08d-6a1002d9bf68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.471408 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4c886d2c-c3da-4afb-a08d-6a1002d9bf68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.475916 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1279/2553] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:27 2022] DELETE /identity/v3/projects/d7e277b0055d4b1d8e1c48a60347b38b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:27.475916 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c886d2c-c3da-4afb-a08d-6a1002d9bf68 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:27.477342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4c886d2c-c3da-4afb-a08d-6a1002d9bf68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:27.494762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:27.536934 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742e77e-edd7-429c-8dcc-a1081045c656 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:27.548834 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c886d2c-c3da-4afb-a08d-6a1002d9bf68 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:27.553622 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f78a5-b315-4833-b200-b5794a63598c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:27.554257 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f78a5-b315-4833-b200-b5794a63598c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:27.554542 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f78a5-b315-4833-b200-b5794a63598c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:27.554822 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f78a5-b315-4833-b200-b5794a63598c admin admin] PATH_INFO: `/v3/users/0d0868a9d6494a4e95c236cf9483ac5d` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:27.555981 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f78a5-b315-4833-b200-b5794a63598c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d0868a9d6494a4e95c236cf9483ac5d)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:27.557926 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f78a5-b315-4833-b200-b5794a63598c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:27.570953 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:27.599540 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5670ccf-285d-4671-aee1-b3298cc0eb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:27.600215 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5670ccf-285d-4671-aee1-b3298cc0eb1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:27.600499 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5670ccf-285d-4671-aee1-b3298cc0eb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:27.600778 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5670ccf-285d-4671-aee1-b3298cc0eb1a admin admin] PATH_INFO: `/v3/projects/3c6482bb91c24939ba18c03a3e5bb18c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:27.601914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5670ccf-285d-4671-aee1-b3298cc0eb1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c6482bb91c24939ba18c03a3e5bb18c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:27.603989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5670ccf-285d-4671-aee1-b3298cc0eb1a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:27.609515 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f94f78a5-b315-4833-b200-b5794a63598c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d0868a9d6494a4e95c236cf9483ac5d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.610220 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f94f78a5-b315-4833-b200-b5794a63598c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d0868a9d6494a4e95c236cf9483ac5d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.617590 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f94f78a5-b315-4833-b200-b5794a63598c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d0868a9d6494a4e95c236cf9483ac5d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.625709 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f94f78a5-b315-4833-b200-b5794a63598c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d0868a9d6494a4e95c236cf9483ac5d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.626500 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f94f78a5-b315-4833-b200-b5794a63598c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d0868a9d6494a4e95c236cf9483ac5d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.645924 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1280/2554] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:27 2022] DELETE /identity/v3/users/0d0868a9d6494a4e95c236cf9483ac5d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:27.647294 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f78a5-b315-4833-b200-b5794a63598c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:27.651916 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f94f78a5-b315-4833-b200-b5794a63598c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:27.654490 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b5670ccf-285d-4671-aee1-b3298cc0eb1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c6482bb91c24939ba18c03a3e5bb18c', 'request_id': 'req-b5670ccf-285d-4671-aee1-b3298cc0eb1a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.663884 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b5670ccf-285d-4671-aee1-b3298cc0eb1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c6482bb91c24939ba18c03a3e5bb18c', 'request_id': 'req-b5670ccf-285d-4671-aee1-b3298cc0eb1a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.671766 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b5670ccf-285d-4671-aee1-b3298cc0eb1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c6482bb91c24939ba18c03a3e5bb18c', 'request_id': 'req-b5670ccf-285d-4671-aee1-b3298cc0eb1a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.683411 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b5670ccf-285d-4671-aee1-b3298cc0eb1a admin admin] The token cache is being invalidate because project 3c6482bb91c24939ba18c03a3e5bb18c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:27.683797 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b5670ccf-285d-4671-aee1-b3298cc0eb1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.684599 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b5670ccf-285d-4671-aee1-b3298cc0eb1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.687083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1275/2555] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:27 2022] DELETE /identity/v3/projects/3c6482bb91c24939ba18c03a3e5bb18c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:27.692298 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7d386-d918-4603-ad9c-3732db140b53 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:27.692589 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7d386-d918-4603-ad9c-3732db140b53 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:27.692881 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7d386-d918-4603-ad9c-3732db140b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:27.715244 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-dec7d386-d918-4603-ad9c-3732db140b53 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:27.716661 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dec7d386-d918-4603-ad9c-3732db140b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:27.730854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f78a5-b315-4833-b200-b5794a63598c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:27.737034 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b80637-97a3-401a-b786-bf88f13280a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:27.737805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b80637-97a3-401a-b786-bf88f13280a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:27.738094 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b80637-97a3-401a-b786-bf88f13280a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:27.738373 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b80637-97a3-401a-b786-bf88f13280a5 admin admin] PATH_INFO: `/v3/users/0a3ae2fb249c4c9ab17e07928974cf09` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:27.740007 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b80637-97a3-401a-b786-bf88f13280a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a3ae2fb249c4c9ab17e07928974cf09)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:27.742211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b80637-97a3-401a-b786-bf88f13280a5 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:27.790575 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1276/2556] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:16:27.800013 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a230a-4c54-4c10-8b77-0a9f4e33f922 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:27.800305 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a230a-4c54-4c10-8b77-0a9f4e33f922 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:27.800599 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a230a-4c54-4c10-8b77-0a9f4e33f922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:27.806399 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-30b80637-97a3-401a-b786-bf88f13280a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a3ae2fb249c4c9ab17e07928974cf09'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.807341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-30b80637-97a3-401a-b786-bf88f13280a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a3ae2fb249c4c9ab17e07928974cf09'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.814326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:27.818029 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-30b80637-97a3-401a-b786-bf88f13280a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a3ae2fb249c4c9ab17e07928974cf09'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.826894 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-30b80637-97a3-401a-b786-bf88f13280a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a3ae2fb249c4c9ab17e07928974cf09'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.827876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-30b80637-97a3-401a-b786-bf88f13280a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a3ae2fb249c4c9ab17e07928974cf09'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:27.832267 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-263a230a-4c54-4c10-8b77-0a9f4e33f922 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:27.834228 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-263a230a-4c54-4c10-8b77-0a9f4e33f922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:27.843594 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1277/2557] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:16:27.847398 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a230a-4c54-4c10-8b77-0a9f4e33f922 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:27.851391 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1281/2558] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:27 2022] DELETE /identity/v3/users/0a3ae2fb249c4c9ab17e07928974cf09 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:27.857229 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a230a-4c54-4c10-8b77-0a9f4e33f922 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:27.930945 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72962ac-7bd8-4859-ba15-d4f9851a8ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:27.932000 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72962ac-7bd8-4859-ba15-d4f9851a8ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:27.932000 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72962ac-7bd8-4859-ba15-d4f9851a8ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:27.932234 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72962ac-7bd8-4859-ba15-d4f9851a8ce1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:27.932856 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72962ac-7bd8-4859-ba15-d4f9851a8ce1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:16:27.933169 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72962ac-7bd8-4859-ba15-d4f9851a8ce1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:27.934865 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72962ac-7bd8-4859-ba15-d4f9851a8ce1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:27.949125 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1278/2559] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:16:27.954032 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b80637-97a3-401a-b786-bf88f13280a5 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:27.958672 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72962ac-7bd8-4859-ba15-d4f9851a8ce1 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:27.960440 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e72962ac-7bd8-4859-ba15-d4f9851a8ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:27.964815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b80637-97a3-401a-b786-bf88f13280a5 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:27.971256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0654ab-97c3-4e1f-846a-f49cb5c0f312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:27.972265 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0654ab-97c3-4e1f-846a-f49cb5c0f312 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:27.972742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0654ab-97c3-4e1f-846a-f49cb5c0f312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:27.973121 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0654ab-97c3-4e1f-846a-f49cb5c0f312 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:27.973848 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0654ab-97c3-4e1f-846a-f49cb5c0f312 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:27.975321 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72962ac-7bd8-4859-ba15-d4f9851a8ce1 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:27.976013 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0654ab-97c3-4e1f-846a-f49cb5c0f312 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:27.980962 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd1385e-049e-4de0-bd5d-b9f43c6be9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:27.981662 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1385e-049e-4de0-bd5d-b9f43c6be9c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:27.981951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1385e-049e-4de0-bd5d-b9f43c6be9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:27.982228 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1385e-049e-4de0-bd5d-b9f43c6be9c6 admin admin] PATH_INFO: `/v3/projects/07ee9a9e4c89439497de43cc49da826d` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:27.983223 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:27.984041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd1385e-049e-4de0-bd5d-b9f43c6be9c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07ee9a9e4c89439497de43cc49da826d)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:27.985981 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd1385e-049e-4de0-bd5d-b9f43c6be9c6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:28.021000 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1282/2560] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:27 2022] POST /identity/v3/projects => generated 365 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:16:28.022432 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0654ab-97c3-4e1f-846a-f49cb5c0f312 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:28.066927 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3cd1385e-049e-4de0-bd5d-b9f43c6be9c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07ee9a9e4c89439497de43cc49da826d', 'request_id': 'req-3cd1385e-049e-4de0-bd5d-b9f43c6be9c6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:28.079005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3cd1385e-049e-4de0-bd5d-b9f43c6be9c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07ee9a9e4c89439497de43cc49da826d', 'request_id': 'req-3cd1385e-049e-4de0-bd5d-b9f43c6be9c6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:28.090509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3cd1385e-049e-4de0-bd5d-b9f43c6be9c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07ee9a9e4c89439497de43cc49da826d', 'request_id': 'req-3cd1385e-049e-4de0-bd5d-b9f43c6be9c6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:28.111520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3cd1385e-049e-4de0-bd5d-b9f43c6be9c6 admin admin] The token cache is being invalidate because project 07ee9a9e4c89439497de43cc49da826d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:28.112032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3cd1385e-049e-4de0-bd5d-b9f43c6be9c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:28.113097 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-3cd1385e-049e-4de0-bd5d-b9f43c6be9c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:28.114861 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1279/2561] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:27 2022] DELETE /identity/v3/projects/07ee9a9e4c89439497de43cc49da826d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:16:28.120335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9557ed7-144c-4d93-9126-03e6e35c178a None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:28.120814 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9557ed7-144c-4d93-9126-03e6e35c178a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:28.121318 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9557ed7-144c-4d93-9126-03e6e35c178a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:28.121643 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0654ab-97c3-4e1f-846a-f49cb5c0f312 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:28.144956 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c9557ed7-144c-4d93-9126-03e6e35c178a None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:28.146489 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c9557ed7-144c-4d93-9126-03e6e35c178a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:28.205816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc8cd6-ba29-405d-a127-476085470a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:28.206510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc8cd6-ba29-405d-a127-476085470a9e admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:28.206861 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc8cd6-ba29-405d-a127-476085470a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:28.207204 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc8cd6-ba29-405d-a127-476085470a9e admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:28.207986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc8cd6-ba29-405d-a127-476085470a9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:28.210267 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bc8cd6-ba29-405d-a127-476085470a9e admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:28.234791 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1280/2562] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:28.235498 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1283/2563] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:28 2022] POST /identity/v3/users => generated 283 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:28.235697 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9557ed7-144c-4d93-9126-03e6e35c178a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:28.237107 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c9557ed7-144c-4d93-9126-03e6e35c178a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:28.242802 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40715eef-2ab0-4862-843e-f189d8e2b5db None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:28.243150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40715eef-2ab0-4862-843e-f189d8e2b5db None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:28.243685 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40715eef-2ab0-4862-843e-f189d8e2b5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:28.249325 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9557ed7-144c-4d93-9126-03e6e35c178a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:28.254461 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bf343-87ce-4a7f-a699-71171d2a9ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:28.255083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bf343-87ce-4a7f-a699-71171d2a9ec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:28.255363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bf343-87ce-4a7f-a699-71171d2a9ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:28.255734 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bf343-87ce-4a7f-a699-71171d2a9ec6 admin admin] PATH_INFO: `/v3/projects/28e918f14a6b47338303abb0d3670926` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:28.256873 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762bf343-87ce-4a7f-a699-71171d2a9ec6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28e918f14a6b47338303abb0d3670926)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:28.259046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762bf343-87ce-4a7f-a699-71171d2a9ec6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:28.265851 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-40715eef-2ab0-4862-843e-f189d8e2b5db None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:28.267168 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-40715eef-2ab0-4862-843e-f189d8e2b5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:28.268662 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:28.276024 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1284/2564] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:28.280809 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40715eef-2ab0-4862-843e-f189d8e2b5db None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:28.282163 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-40715eef-2ab0-4862-843e-f189d8e2b5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:28.294285 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40715eef-2ab0-4862-843e-f189d8e2b5db None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:28.299082 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b7ad5-2c28-47e2-bb6d-4b90a10f884c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:28.299754 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b7ad5-2c28-47e2-bb6d-4b90a10f884c admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:28.300030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b7ad5-2c28-47e2-bb6d-4b90a10f884c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:28.300307 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b7ad5-2c28-47e2-bb6d-4b90a10f884c admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:28.300952 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b7ad5-2c28-47e2-bb6d-4b90a10f884c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:28.302563 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b7ad5-2c28-47e2-bb6d-4b90a10f884c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:28.310415 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-762bf343-87ce-4a7f-a699-71171d2a9ec6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28e918f14a6b47338303abb0d3670926', 'request_id': 'req-762bf343-87ce-4a7f-a699-71171d2a9ec6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:28.316183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1285/2565] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:28.316183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b7ad5-2c28-47e2-bb6d-4b90a10f884c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:28.319725 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-762bf343-87ce-4a7f-a699-71171d2a9ec6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28e918f14a6b47338303abb0d3670926', 'request_id': 'req-762bf343-87ce-4a7f-a699-71171d2a9ec6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:28.326207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-762bf343-87ce-4a7f-a699-71171d2a9ec6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28e918f14a6b47338303abb0d3670926', 'request_id': 'req-762bf343-87ce-4a7f-a699-71171d2a9ec6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:28.326589 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b7ad5-2c28-47e2-bb6d-4b90a10f884c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:28.340962 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-762bf343-87ce-4a7f-a699-71171d2a9ec6 admin admin] The token cache is being invalidate because project 28e918f14a6b47338303abb0d3670926 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:28.341253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-762bf343-87ce-4a7f-a699-71171d2a9ec6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:28.341925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-762bf343-87ce-4a7f-a699-71171d2a9ec6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:28.344173 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1281/2566] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:28 2022] DELETE /identity/v3/projects/28e918f14a6b47338303abb0d3670926 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:28.344943 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bf343-87ce-4a7f-a699-71171d2a9ec6 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:28.346247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-762bf343-87ce-4a7f-a699-71171d2a9ec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:28.386609 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c44109-5a06-46bb-93b8-75f658bc6c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:28.387224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c44109-5a06-46bb-93b8-75f658bc6c44 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:28.387508 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c44109-5a06-46bb-93b8-75f658bc6c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:28.387821 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c44109-5a06-46bb-93b8-75f658bc6c44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:28.388467 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c44109-5a06-46bb-93b8-75f658bc6c44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:16:28.388775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c44109-5a06-46bb-93b8-75f658bc6c44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:28.390956 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c44109-5a06-46bb-93b8-75f658bc6c44 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:28.400037 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:28.404307 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1286/2567] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:16:28.406882 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a4a6b-bdbd-4f18-a6b1-1055254b9cb9 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:28.407606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a4a6b-bdbd-4f18-a6b1-1055254b9cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:28.408907 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a4a6b-bdbd-4f18-a6b1-1055254b9cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:28.412617 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bf343-87ce-4a7f-a699-71171d2a9ec6 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:28.417378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131551a9-61c0-4dc4-bbd0-719be2e40742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:28.417989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131551a9-61c0-4dc4-bbd0-719be2e40742 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:28.418273 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131551a9-61c0-4dc4-bbd0-719be2e40742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:28.418550 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131551a9-61c0-4dc4-bbd0-719be2e40742 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:28.419196 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131551a9-61c0-4dc4-bbd0-719be2e40742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:28.420891 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131551a9-61c0-4dc4-bbd0-719be2e40742 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:28.428303 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-499a4a6b-bdbd-4f18-a6b1-1055254b9cb9 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:28.429501 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-499a4a6b-bdbd-4f18-a6b1-1055254b9cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:28.434257 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1282/2568] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:28.434257 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131551a9-61c0-4dc4-bbd0-719be2e40742 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:28.434257 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-131551a9-61c0-4dc4-bbd0-719be2e40742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:28.441320 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1287/2569] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:28.442176 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a4a6b-bdbd-4f18-a6b1-1055254b9cb9 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:28.451418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131551a9-61c0-4dc4-bbd0-719be2e40742 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:28.451642 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a4a6b-bdbd-4f18-a6b1-1055254b9cb9 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:28.456567 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e6aa4-8006-4d88-a501-a9979ee89adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:28.456783 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d0302-5551-4f7b-98ad-a4dc6649b85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:28.457170 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e6aa4-8006-4d88-a501-a9979ee89adc admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:28.457425 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d0302-5551-4f7b-98ad-a4dc6649b85c admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:28.457535 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e6aa4-8006-4d88-a501-a9979ee89adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:28.457706 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d0302-5551-4f7b-98ad-a4dc6649b85c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:28.457955 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e6aa4-8006-4d88-a501-a9979ee89adc admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:28.458062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d0302-5551-4f7b-98ad-a4dc6649b85c admin admin] PATH_INFO: `/v3/projects/1d348cdc67014251875439ca27223e80/users/90bcc056619146808e0b52b67614ed33/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:28.458598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e6aa4-8006-4d88-a501-a9979ee89adc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:28.460558 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e6aa4-8006-4d88-a501-a9979ee89adc admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:28.475134 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d0302-5551-4f7b-98ad-a4dc6649b85c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d348cdc67014251875439ca27223e80, user_id=90bcc056619146808e0b52b67614ed33, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:28.478715 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d0302-5551-4f7b-98ad-a4dc6649b85c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:28.490579 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1283/2570] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:28 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:28.490579 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5015cf-d65b-48d5-a00a-b6c5ea715e2e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:28.490579 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5015cf-d65b-48d5-a00a-b6c5ea715e2e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:28.490579 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5015cf-d65b-48d5-a00a-b6c5ea715e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:28.494452 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1288/2571] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:28 2022] PUT /identity/v3/projects/1d348cdc67014251875439ca27223e80/users/90bcc056619146808e0b52b67614ed33/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:28.494978 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d0302-5551-4f7b-98ad-a4dc6649b85c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:28.515585 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0d5015cf-d65b-48d5-a00a-b6c5ea715e2e None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:28.516803 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0d5015cf-d65b-48d5-a00a-b6c5ea715e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:28.563327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d0302-5551-4f7b-98ad-a4dc6649b85c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:28.576214 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1284/2572] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:28.578005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c039b02-3caf-4718-a5e9-0a47f92d37b1 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:28.578571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c039b02-3caf-4718-a5e9-0a47f92d37b1 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:28.578963 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c039b02-3caf-4718-a5e9-0a47f92d37b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:28.599264 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:28.607979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7c039b02-3caf-4718-a5e9-0a47f92d37b1 None None] MFA Rules not processed for user `90bcc056619146808e0b52b67614ed33`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:28.609993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7c039b02-3caf-4718-a5e9-0a47f92d37b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:28.622749 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17c9b5-73f2-4bb1-a535-75195beaf697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:28.623363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17c9b5-73f2-4bb1-a535-75195beaf697 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:28.623684 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17c9b5-73f2-4bb1-a535-75195beaf697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:28.624510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17c9b5-73f2-4bb1-a535-75195beaf697 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:28.625151 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17c9b5-73f2-4bb1-a535-75195beaf697 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:28.627119 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17c9b5-73f2-4bb1-a535-75195beaf697 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:28.649772 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1289/2573] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:28 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:28.660362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17c9b5-73f2-4bb1-a535-75195beaf697 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:28.672479 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17c9b5-73f2-4bb1-a535-75195beaf697 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:28.677700 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c65f3e3-b2ee-4b38-8d92-dc7f556fcdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:28.678392 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65f3e3-b2ee-4b38-8d92-dc7f556fcdb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:28.679013 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65f3e3-b2ee-4b38-8d92-dc7f556fcdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:28.679373 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65f3e3-b2ee-4b38-8d92-dc7f556fcdb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:28.680061 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c65f3e3-b2ee-4b38-8d92-dc7f556fcdb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:16:28.680376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c65f3e3-b2ee-4b38-8d92-dc7f556fcdb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:28.681990 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c65f3e3-b2ee-4b38-8d92-dc7f556fcdb6 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:28.682133 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1285/2574] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:28.682976 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c039b02-3caf-4718-a5e9-0a47f92d37b1 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:28.694799 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c039b02-3caf-4718-a5e9-0a47f92d37b1 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:28.698818 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1290/2575] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:16:28.698818 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c65f3e3-b2ee-4b38-8d92-dc7f556fcdb6 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:28.703363 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f38964-11eb-48ac-b648-45c0fd803c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:28.703786 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f38964-11eb-48ac-b648-45c0fd803c7e admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:28.704065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f38964-11eb-48ac-b648-45c0fd803c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:28.704338 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f38964-11eb-48ac-b648-45c0fd803c7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:28.704980 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f38964-11eb-48ac-b648-45c0fd803c7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:28.706632 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f38964-11eb-48ac-b648-45c0fd803c7e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:28.708650 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c65f3e3-b2ee-4b38-8d92-dc7f556fcdb6 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:28.713382 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743451f-82f6-4342-995d-f950df5c24cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:28.713998 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743451f-82f6-4342-995d-f950df5c24cd admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:28.714902 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743451f-82f6-4342-995d-f950df5c24cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:28.715192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743451f-82f6-4342-995d-f950df5c24cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:28.715855 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743451f-82f6-4342-995d-f950df5c24cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:28.719820 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1286/2576] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:28.719820 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f38964-11eb-48ac-b648-45c0fd803c7e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:28.720075 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743451f-82f6-4342-995d-f950df5c24cd admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:28.727930 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f38964-11eb-48ac-b648-45c0fd803c7e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:28.732184 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a647314-d0ae-4a17-9240-abc02e981aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:28.732771 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a647314-d0ae-4a17-9240-abc02e981aab admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:28.733050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a647314-d0ae-4a17-9240-abc02e981aab admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:28.733419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a647314-d0ae-4a17-9240-abc02e981aab admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:28.734178 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a647314-d0ae-4a17-9240-abc02e981aab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:28.736102 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a647314-d0ae-4a17-9240-abc02e981aab admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:28.745337 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1291/2577] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:28 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:28.747191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743451f-82f6-4342-995d-f950df5c24cd admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:28.767844 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1287/2578] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:28 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:28.767844 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a647314-d0ae-4a17-9240-abc02e981aab admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:28.783642 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:28.813690 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743451f-82f6-4342-995d-f950df5c24cd admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:28.829541 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a647314-d0ae-4a17-9240-abc02e981aab admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:28.834653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2445e52-179e-4d82-9c3c-2eda4a6119df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:28.835338 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2445e52-179e-4d82-9c3c-2eda4a6119df admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:28.835726 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2445e52-179e-4d82-9c3c-2eda4a6119df admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:28.836071 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2445e52-179e-4d82-9c3c-2eda4a6119df admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:28.836799 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2445e52-179e-4d82-9c3c-2eda4a6119df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:28.839187 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2445e52-179e-4d82-9c3c-2eda4a6119df admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:28.859982 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1288/2579] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:28 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:28.863779 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2445e52-179e-4d82-9c3c-2eda4a6119df admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:28.870689 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b831d5-0b5a-4419-b721-2bb08d65b016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:28.871302 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b831d5-0b5a-4419-b721-2bb08d65b016 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:28.871606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b831d5-0b5a-4419-b721-2bb08d65b016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:28.871890 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b831d5-0b5a-4419-b721-2bb08d65b016 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:28.872528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b831d5-0b5a-4419-b721-2bb08d65b016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:28.873130 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2445e52-179e-4d82-9c3c-2eda4a6119df admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:28.874655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b831d5-0b5a-4419-b721-2bb08d65b016 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:28.878691 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5584cc08-4402-4301-b77d-8e13bb0ca1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:28.879270 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584cc08-4402-4301-b77d-8e13bb0ca1e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:28.879547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584cc08-4402-4301-b77d-8e13bb0ca1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:28.879844 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584cc08-4402-4301-b77d-8e13bb0ca1e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:28.880461 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5584cc08-4402-4301-b77d-8e13bb0ca1e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:28.882350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5584cc08-4402-4301-b77d-8e13bb0ca1e7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:28.885006 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1292/2580] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:28.885808 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b831d5-0b5a-4419-b721-2bb08d65b016 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:28.895792 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b831d5-0b5a-4419-b721-2bb08d65b016 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:28.901044 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65054e7d-bc9b-4f69-b1b3-f3a0cea114a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:28.901639 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65054e7d-bc9b-4f69-b1b3-f3a0cea114a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:28.901920 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65054e7d-bc9b-4f69-b1b3-f3a0cea114a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:28.902202 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65054e7d-bc9b-4f69-b1b3-f3a0cea114a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:28.902816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65054e7d-bc9b-4f69-b1b3-f3a0cea114a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:28.904424 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65054e7d-bc9b-4f69-b1b3-f3a0cea114a0 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:28.910826 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1289/2581] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:28 2022] POST /identity/v3/users => generated 293 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:28.910826 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5584cc08-4402-4301-b77d-8e13bb0ca1e7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:28.914971 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1293/2582] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:28.915827 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65054e7d-bc9b-4f69-b1b3-f3a0cea114a0 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:28.920608 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5584cc08-4402-4301-b77d-8e13bb0ca1e7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:28.925578 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd7189-3cc8-44a2-b846-053c3f9b0d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:28.925743 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65054e7d-bc9b-4f69-b1b3-f3a0cea114a0 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:28.926232 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd7189-3cc8-44a2-b846-053c3f9b0d03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:28.926583 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd7189-3cc8-44a2-b846-053c3f9b0d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:28.926868 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd7189-3cc8-44a2-b846-053c3f9b0d03 admin admin] PATH_INFO: `/v3/projects/c4a21c7763de40b2a48c87cd5b70bd57/users/fba1e335998f4e518dea0102e7ff3243/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:28.930565 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e6886-ff0d-48c9-8b1c-b1e4fa6c6473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:28.931314 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e6886-ff0d-48c9-8b1c-b1e4fa6c6473 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:28.931626 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e6886-ff0d-48c9-8b1c-b1e4fa6c6473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:28.931916 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e6886-ff0d-48c9-8b1c-b1e4fa6c6473 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:28.932540 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e6886-ff0d-48c9-8b1c-b1e4fa6c6473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:28.934170 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e6886-ff0d-48c9-8b1c-b1e4fa6c6473 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:28.943085 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd7189-3cc8-44a2-b846-053c3f9b0d03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4a21c7763de40b2a48c87cd5b70bd57, user_id=fba1e335998f4e518dea0102e7ff3243, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:28.945773 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd7189-3cc8-44a2-b846-053c3f9b0d03 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:28.946435 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1294/2583] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:28.947353 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e6886-ff0d-48c9-8b1c-b1e4fa6c6473 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:28.958525 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e6886-ff0d-48c9-8b1c-b1e4fa6c6473 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:28.963082 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1290/2584] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:28 2022] PUT /identity/v3/projects/c4a21c7763de40b2a48c87cd5b70bd57/users/fba1e335998f4e518dea0102e7ff3243/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:28.964013 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd7189-3cc8-44a2-b846-053c3f9b0d03 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:28.981244 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:29.024843 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c868c9-b88f-43f6-84c7-1cfc6f150a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:29.025499 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c868c9-b88f-43f6-84c7-1cfc6f150a5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:29.025777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c868c9-b88f-43f6-84c7-1cfc6f150a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:29.026051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c868c9-b88f-43f6-84c7-1cfc6f150a5d admin admin] PATH_INFO: `/v3/projects/fc1807a0636c45b0b7ef235eb3cf4f3d/users/b12daa55c1a34e8189d7cef265efa86d/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:29.029971 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd7189-3cc8-44a2-b846-053c3f9b0d03 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:29.040869 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0ee47-d1ba-4182-b8e7-35b2bd1ba3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:29.041455 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c868c9-b88f-43f6-84c7-1cfc6f150a5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc1807a0636c45b0b7ef235eb3cf4f3d, user_id=b12daa55c1a34e8189d7cef265efa86d, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:29.041587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0ee47-d1ba-4182-b8e7-35b2bd1ba3ab admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:29.041766 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0ee47-d1ba-4182-b8e7-35b2bd1ba3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:29.042043 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0ee47-d1ba-4182-b8e7-35b2bd1ba3ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:29.042699 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d0ee47-d1ba-4182-b8e7-35b2bd1ba3ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:29.044250 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c868c9-b88f-43f6-84c7-1cfc6f150a5d admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:29.044507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d0ee47-d1ba-4182-b8e7-35b2bd1ba3ab admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:29.056702 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1291/2585] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:29.057918 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f00bfc-144d-418d-9d12-57c42205140f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:29.058204 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f00bfc-144d-418d-9d12-57c42205140f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:29.058478 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f00bfc-144d-418d-9d12-57c42205140f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:29.062239 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1295/2586] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:28 2022] PUT /identity/v3/projects/fc1807a0636c45b0b7ef235eb3cf4f3d/users/b12daa55c1a34e8189d7cef265efa86d/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:29.063038 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c868c9-b88f-43f6-84c7-1cfc6f150a5d admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:29.088136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-11f00bfc-144d-418d-9d12-57c42205140f None None] MFA Rules not processed for user `fba1e335998f4e518dea0102e7ff3243`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:29.089369 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-11f00bfc-144d-418d-9d12-57c42205140f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:29.103141 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:29.130489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c868c9-b88f-43f6-84c7-1cfc6f150a5d admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:29.140173 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de9598-3cb6-4582-aea7-dbf831d60fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:29.140906 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de9598-3cb6-4582-aea7-dbf831d60fc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:29.141182 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de9598-3cb6-4582-aea7-dbf831d60fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:29.141453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de9598-3cb6-4582-aea7-dbf831d60fc1 admin admin] PATH_INFO: `/v3/projects/053dfa24a9884cbeb7afc6e017d02e80/users/a810826b0d62473481cccf432a7d5dbb/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:29.156719 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1292/2587] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:29.157572 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f00bfc-144d-418d-9d12-57c42205140f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:29.160963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de9598-3cb6-4582-aea7-dbf831d60fc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=053dfa24a9884cbeb7afc6e017d02e80, user_id=a810826b0d62473481cccf432a7d5dbb, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:29.163694 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de9598-3cb6-4582-aea7-dbf831d60fc1 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:29.168222 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f00bfc-144d-418d-9d12-57c42205140f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:29.174141 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f36a15e-baa2-4df2-9f84-b390c6627832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:29.174823 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36a15e-baa2-4df2-9f84-b390c6627832 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:29.175015 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36a15e-baa2-4df2-9f84-b390c6627832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:29.175291 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36a15e-baa2-4df2-9f84-b390c6627832 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:29.175963 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f36a15e-baa2-4df2-9f84-b390c6627832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:29.177557 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f36a15e-baa2-4df2-9f84-b390c6627832 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:29.177670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1296/2588] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:29 2022] PUT /identity/v3/projects/053dfa24a9884cbeb7afc6e017d02e80/users/a810826b0d62473481cccf432a7d5dbb/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:29.178622 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de9598-3cb6-4582-aea7-dbf831d60fc1 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:29.197990 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1293/2589] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:29.199355 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d59a25-4a94-4cee-8184-86c067f71f98 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:29.199739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d59a25-4a94-4cee-8184-86c067f71f98 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:29.200083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d59a25-4a94-4cee-8184-86c067f71f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:29.228544 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-59d59a25-4a94-4cee-8184-86c067f71f98 None None] MFA Rules not processed for user `a810826b0d62473481cccf432a7d5dbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:29.229683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-59d59a25-4a94-4cee-8184-86c067f71f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:29.242510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de9598-3cb6-4582-aea7-dbf831d60fc1 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:29.247955 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f2903-c443-4892-beff-daf97876742d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:29.248559 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f2903-c443-4892-beff-daf97876742d admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:29.248861 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f2903-c443-4892-beff-daf97876742d admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:29.249859 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f2903-c443-4892-beff-daf97876742d admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:29.250780 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f2903-c443-4892-beff-daf97876742d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:29.252734 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f2903-c443-4892-beff-daf97876742d admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:29.281435 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1297/2590] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:29 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:29.282820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f2903-c443-4892-beff-daf97876742d admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:29.303580 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1294/2591] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:29.307078 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d59a25-4a94-4cee-8184-86c067f71f98 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:29.315417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d59a25-4a94-4cee-8184-86c067f71f98 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:29.332216 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:29.352147 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f2903-c443-4892-beff-daf97876742d admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:29.356900 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28412613-e112-4b2c-863d-081ef3167920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:29.357563 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28412613-e112-4b2c-863d-081ef3167920 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:29.357849 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28412613-e112-4b2c-863d-081ef3167920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:29.358124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28412613-e112-4b2c-863d-081ef3167920 admin admin] PATH_INFO: `/v3/projects/fc1807a0636c45b0b7ef235eb3cf4f3d/users/b12daa55c1a34e8189d7cef265efa86d/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:29.361391 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28412613-e112-4b2c-863d-081ef3167920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc1807a0636c45b0b7ef235eb3cf4f3d, user_id=b12daa55c1a34e8189d7cef265efa86d, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:29.364077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28412613-e112-4b2c-863d-081ef3167920 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:29.374112 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75470210-2c3f-4aa8-8f54-61ecf47bc554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:29.374726 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75470210-2c3f-4aa8-8f54-61ecf47bc554 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:29.375012 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75470210-2c3f-4aa8-8f54-61ecf47bc554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:29.375208 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1298/2592] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:29 2022] PUT /identity/v3/projects/fc1807a0636c45b0b7ef235eb3cf4f3d/users/b12daa55c1a34e8189d7cef265efa86d/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:29.375352 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75470210-2c3f-4aa8-8f54-61ecf47bc554 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:29.375944 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75470210-2c3f-4aa8-8f54-61ecf47bc554 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:29.376313 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28412613-e112-4b2c-863d-081ef3167920 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:29.377947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75470210-2c3f-4aa8-8f54-61ecf47bc554 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:29.403129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1295/2593] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:29 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:29.403129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8c913-c324-4c18-9a73-bf3a8150d032 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:29.403129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8c913-c324-4c18-9a73-bf3a8150d032 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:29.403129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8c913-c324-4c18-9a73-bf3a8150d032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:29.424508 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:29.431394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-02d8c913-c324-4c18-9a73-bf3a8150d032 None None] MFA Rules not processed for user `b12daa55c1a34e8189d7cef265efa86d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:29.432659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-02d8c913-c324-4c18-9a73-bf3a8150d032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:29.446232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28412613-e112-4b2c-863d-081ef3167920 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:29.450920 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd79700-d10f-4530-9b6a-7682ac779d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:29.451528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd79700-d10f-4530-9b6a-7682ac779d80 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:29.451838 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd79700-d10f-4530-9b6a-7682ac779d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:29.452319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd79700-d10f-4530-9b6a-7682ac779d80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:29.453069 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd79700-d10f-4530-9b6a-7682ac779d80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:29.454972 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd79700-d10f-4530-9b6a-7682ac779d80 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:29.486905 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1299/2594] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:29 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:29.487789 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd79700-d10f-4530-9b6a-7682ac779d80 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:29.505028 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1296/2595] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:29 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:29.509457 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8c913-c324-4c18-9a73-bf3a8150d032 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:29.517865 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8c913-c324-4c18-9a73-bf3a8150d032 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:29.551525 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd79700-d10f-4530-9b6a-7682ac779d80 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:29.556341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a2663-4a4e-49f3-b781-0938b3e87abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:29.556951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a2663-4a4e-49f3-b781-0938b3e87abf admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:29.557235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a2663-4a4e-49f3-b781-0938b3e87abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:29.557525 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a2663-4a4e-49f3-b781-0938b3e87abf admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:29.558161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a2663-4a4e-49f3-b781-0938b3e87abf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:29.559850 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a2663-4a4e-49f3-b781-0938b3e87abf admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:29.569941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1300/2596] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:29.569941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a2663-4a4e-49f3-b781-0938b3e87abf admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:29.569941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-1e4a2663-4a4e-49f3-b781-0938b3e87abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:29.579259 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d7280-6cac-4ed0-b4f3-de90013cf5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:29.579911 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d7280-6cac-4ed0-b4f3-de90013cf5cb admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:29.580194 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d7280-6cac-4ed0-b4f3-de90013cf5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:29.580496 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d7280-6cac-4ed0-b4f3-de90013cf5cb admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:29.581250 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d7280-6cac-4ed0-b4f3-de90013cf5cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:29.583269 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d7280-6cac-4ed0-b4f3-de90013cf5cb admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:29.605771 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1297/2597] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:29 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:29.606578 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d7280-6cac-4ed0-b4f3-de90013cf5cb admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:29.616861 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d7280-6cac-4ed0-b4f3-de90013cf5cb admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:29.621584 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087c0c1e-de75-4950-9971-bef83eaabc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:29.622182 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c0c1e-de75-4950-9971-bef83eaabc96 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:29.622466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c0c1e-de75-4950-9971-bef83eaabc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:29.622741 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c0c1e-de75-4950-9971-bef83eaabc96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:29.623378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087c0c1e-de75-4950-9971-bef83eaabc96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:29.625492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087c0c1e-de75-4950-9971-bef83eaabc96 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:29.635177 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a2663-4a4e-49f3-b781-0938b3e87abf admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:29.636417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1298/2598] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:29.637301 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087c0c1e-de75-4950-9971-bef83eaabc96 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:29.638977 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd224b0-9c92-4fdd-8e53-bd392aed37a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:29.639780 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd224b0-9c92-4fdd-8e53-bd392aed37a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:29.640336 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd224b0-9c92-4fdd-8e53-bd392aed37a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:29.640687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd224b0-9c92-4fdd-8e53-bd392aed37a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:29.647328 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087c0c1e-de75-4950-9971-bef83eaabc96 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:29.652011 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b16b2c-f1a0-481e-96a3-6f1894943c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:29.652604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b16b2c-f1a0-481e-96a3-6f1894943c84 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:29.652887 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b16b2c-f1a0-481e-96a3-6f1894943c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:29.653164 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b16b2c-f1a0-481e-96a3-6f1894943c84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:29.653801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b16b2c-f1a0-481e-96a3-6f1894943c84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:29.656167 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b16b2c-f1a0-481e-96a3-6f1894943c84 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:29.666920 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1299/2599] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:29.668058 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b16b2c-f1a0-481e-96a3-6f1894943c84 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:29.674100 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:29.680743 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b16b2c-f1a0-481e-96a3-6f1894943c84 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:29.685843 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3cdd8-2e64-4961-899b-ec6c22e2d7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:29.686541 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3cdd8-2e64-4961-899b-ec6c22e2d7b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:29.686883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3cdd8-2e64-4961-899b-ec6c22e2d7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:29.687166 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3cdd8-2e64-4961-899b-ec6c22e2d7b6 admin admin] PATH_INFO: `/v3/projects/d4212d11ac6b4afda1ac59e360d9e51e/users/cc7fcc054fea4baa9301432abe697aa7/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:29.701789 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3cdd8-2e64-4961-899b-ec6c22e2d7b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4212d11ac6b4afda1ac59e360d9e51e, user_id=cc7fcc054fea4baa9301432abe697aa7, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:29.705449 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3cdd8-2e64-4961-899b-ec6c22e2d7b6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:29.709045 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd224b0-9c92-4fdd-8e53-bd392aed37a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:29.710752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd224b0-9c92-4fdd-8e53-bd392aed37a2 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:29.720316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1300/2600] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:29 2022] PUT /identity/v3/projects/d4212d11ac6b4afda1ac59e360d9e51e/users/cc7fcc054fea4baa9301432abe697aa7/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:29.721491 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3cdd8-2e64-4961-899b-ec6c22e2d7b6 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:29.725113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1301/2601] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:29 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:29.731829 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25d29a-1480-4960-80e1-2be8548275f8 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:29.732265 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25d29a-1480-4960-80e1-2be8548275f8 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:29.732741 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25d29a-1480-4960-80e1-2be8548275f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:29.759322 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:29.781140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-bf25d29a-1480-4960-80e1-2be8548275f8 None None] MFA Rules not processed for user `cc7fcc054fea4baa9301432abe697aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:29.782424 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bf25d29a-1480-4960-80e1-2be8548275f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:29.797509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3cdd8-2e64-4961-899b-ec6c22e2d7b6 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:29.810187 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088300f1-e7c1-479f-9841-6db234eb9238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:29.810187 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088300f1-e7c1-479f-9841-6db234eb9238 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:29.810187 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088300f1-e7c1-479f-9841-6db234eb9238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:29.810187 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088300f1-e7c1-479f-9841-6db234eb9238 admin admin] PATH_INFO: `/v3/projects/4a04d886ed7d4f188f7b33e41e94e76f/users/561e1edd6aa142deadbfe4ca5ecb9d4c/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:29.823365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088300f1-e7c1-479f-9841-6db234eb9238 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a04d886ed7d4f188f7b33e41e94e76f, user_id=561e1edd6aa142deadbfe4ca5ecb9d4c, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:29.826151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088300f1-e7c1-479f-9841-6db234eb9238 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:29.842203 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1301/2602] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:29 2022] PUT /identity/v3/projects/4a04d886ed7d4f188f7b33e41e94e76f/users/561e1edd6aa142deadbfe4ca5ecb9d4c/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:29.852329 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088300f1-e7c1-479f-9841-6db234eb9238 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:29.859047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1302/2603] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:16:29.863234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25d29a-1480-4960-80e1-2be8548275f8 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:29.877148 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25d29a-1480-4960-80e1-2be8548275f8 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:29.935106 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088300f1-e7c1-479f-9841-6db234eb9238 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:29.942010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95feadb2-5b25-4aec-9b5b-49cb6b085455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:29.942818 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95feadb2-5b25-4aec-9b5b-49cb6b085455 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:29.943175 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95feadb2-5b25-4aec-9b5b-49cb6b085455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:29.943615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95feadb2-5b25-4aec-9b5b-49cb6b085455 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:29.944253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95feadb2-5b25-4aec-9b5b-49cb6b085455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:29.945969 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95feadb2-5b25-4aec-9b5b-49cb6b085455 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:29.946637 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff20d16-f994-41e1-8a2c-1570694f790b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:29.947252 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff20d16-f994-41e1-8a2c-1570694f790b admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:29.947745 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff20d16-f994-41e1-8a2c-1570694f790b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:29.948020 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff20d16-f994-41e1-8a2c-1570694f790b admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:29.948998 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff20d16-f994-41e1-8a2c-1570694f790b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:29.951048 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff20d16-f994-41e1-8a2c-1570694f790b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:29.961817 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1302/2604] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:29.964645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95feadb2-5b25-4aec-9b5b-49cb6b085455 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:29.976711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95feadb2-5b25-4aec-9b5b-49cb6b085455 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:29.981646 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d24e3-0ff1-42a1-83c7-8c50fb3736d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:29.986152 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1303/2605] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:29 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:16:29.986626 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d24e3-0ff1-42a1-83c7-8c50fb3736d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:29.987124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d24e3-0ff1-42a1-83c7-8c50fb3736d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:29.987274 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff20d16-f994-41e1-8a2c-1570694f790b admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:29.987410 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d24e3-0ff1-42a1-83c7-8c50fb3736d6 admin admin] PATH_INFO: `/v3/projects/4a04d886ed7d4f188f7b33e41e94e76f/users/561e1edd6aa142deadbfe4ca5ecb9d4c/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:29.989672 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d24e3-0ff1-42a1-83c7-8c50fb3736d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a04d886ed7d4f188f7b33e41e94e76f, user_id=561e1edd6aa142deadbfe4ca5ecb9d4c, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:29.992683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d24e3-0ff1-42a1-83c7-8c50fb3736d6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:30.004670 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1303/2606] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:29 2022] PUT /identity/v3/projects/4a04d886ed7d4f188f7b33e41e94e76f/users/561e1edd6aa142deadbfe4ca5ecb9d4c/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:30.014861 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7dfeb9-6a63-4d44-a8a1-21ba091bec83 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:30.015158 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7dfeb9-6a63-4d44-a8a1-21ba091bec83 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:30.015441 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7dfeb9-6a63-4d44-a8a1-21ba091bec83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:30.050592 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-0b7dfeb9-6a63-4d44-a8a1-21ba091bec83 None None] MFA Rules not processed for user `561e1edd6aa142deadbfe4ca5ecb9d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:30.051895 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0b7dfeb9-6a63-4d44-a8a1-21ba091bec83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:30.073409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff20d16-f994-41e1-8a2c-1570694f790b admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:30.098511 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:30.107310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:30.140840 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cab141-d947-435b-bd37-a5f098db9d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:30.141458 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cab141-d947-435b-bd37-a5f098db9d78 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:30.141740 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cab141-d947-435b-bd37-a5f098db9d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:30.142014 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cab141-d947-435b-bd37-a5f098db9d78 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:30.142649 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cab141-d947-435b-bd37-a5f098db9d78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:30.144610 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cab141-d947-435b-bd37-a5f098db9d78 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:30.146910 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1304/2607] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:30 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:30.160993 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7dfeb9-6a63-4d44-a8a1-21ba091bec83 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:30.162716 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0b7dfeb9-6a63-4d44-a8a1-21ba091bec83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:30.174180 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1304/2608] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:29 2022] POST /identity/v3/users => generated 294 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:16:30.180754 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cab141-d947-435b-bd37-a5f098db9d78 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:30.196626 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cab141-d947-435b-bd37-a5f098db9d78 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:30.203179 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4fa17-a6cc-498f-bc74-be553ad0eaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:30.203826 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4fa17-a6cc-498f-bc74-be553ad0eaf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:30.204116 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4fa17-a6cc-498f-bc74-be553ad0eaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:30.204398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4fa17-a6cc-498f-bc74-be553ad0eaf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:30.205050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4fa17-a6cc-498f-bc74-be553ad0eaf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:30.206674 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4fa17-a6cc-498f-bc74-be553ad0eaf5 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:30.218424 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1305/2609] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:30.218424 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4fa17-a6cc-498f-bc74-be553ad0eaf5 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:30.229233 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4fa17-a6cc-498f-bc74-be553ad0eaf5 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:30.229475 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7dfeb9-6a63-4d44-a8a1-21ba091bec83 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:30.233623 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fd9b3-3478-46e9-8847-60049a3b4003 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:30.234155 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e6eeb-4c71-4b82-a62c-3b75d8cbb9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:30.234329 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fd9b3-3478-46e9-8847-60049a3b4003 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:30.234551 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fd9b3-3478-46e9-8847-60049a3b4003 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:30.234767 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e6eeb-4c71-4b82-a62c-3b75d8cbb9c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:30.234869 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fd9b3-3478-46e9-8847-60049a3b4003 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:30.235067 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e6eeb-4c71-4b82-a62c-3b75d8cbb9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:30.235333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e6eeb-4c71-4b82-a62c-3b75d8cbb9c6 admin admin] PATH_INFO: `/v3/users/90bcc056619146808e0b52b67614ed33` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:30.236550 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e6eeb-4c71-4b82-a62c-3b75d8cbb9c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90bcc056619146808e0b52b67614ed33)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:30.238736 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e6eeb-4c71-4b82-a62c-3b75d8cbb9c6 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:30.288351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e89e6eeb-4c71-4b82-a62c-3b75d8cbb9c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90bcc056619146808e0b52b67614ed33'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:30.289023 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e89e6eeb-4c71-4b82-a62c-3b75d8cbb9c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90bcc056619146808e0b52b67614ed33'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:30.296644 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e89e6eeb-4c71-4b82-a62c-3b75d8cbb9c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90bcc056619146808e0b52b67614ed33'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:30.296793 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995fd9b3-3478-46e9-8847-60049a3b4003 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:30.298283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995fd9b3-3478-46e9-8847-60049a3b4003 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:30.303948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e89e6eeb-4c71-4b82-a62c-3b75d8cbb9c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90bcc056619146808e0b52b67614ed33'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:30.304629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-e89e6eeb-4c71-4b82-a62c-3b75d8cbb9c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90bcc056619146808e0b52b67614ed33'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:30.307426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1305/2610] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:16:30.308197 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fd9b3-3478-46e9-8847-60049a3b4003 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:30.319041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fd9b3-3478-46e9-8847-60049a3b4003 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:30.324037 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918be6d-70c9-417d-8061-9341afbfca99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:30.324229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1306/2611] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:30 2022] DELETE /identity/v3/users/90bcc056619146808e0b52b67614ed33 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:30.328286 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918be6d-70c9-417d-8061-9341afbfca99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:30.328623 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918be6d-70c9-417d-8061-9341afbfca99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:30.328859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918be6d-70c9-417d-8061-9341afbfca99 admin admin] PATH_INFO: `/v3/projects/0de7d515d7564e82944640dc066acabf/users/56a7b2ddd7b24a83adc885754dea95c1/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:30.329463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31728ff4-f606-49a9-a92b-e9c20d64995b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:30.329818 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31728ff4-f606-49a9-a92b-e9c20d64995b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:30.330194 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31728ff4-f606-49a9-a92b-e9c20d64995b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:30.347494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918be6d-70c9-417d-8061-9341afbfca99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de7d515d7564e82944640dc066acabf, user_id=56a7b2ddd7b24a83adc885754dea95c1, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:30.350254 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918be6d-70c9-417d-8061-9341afbfca99 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:30.353119 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-31728ff4-f606-49a9-a92b-e9c20d64995b None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:30.354565 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-31728ff4-f606-49a9-a92b-e9c20d64995b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:30.364175 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1306/2612] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:30 2022] PUT /identity/v3/projects/0de7d515d7564e82944640dc066acabf/users/56a7b2ddd7b24a83adc885754dea95c1/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:30.368032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918be6d-70c9-417d-8061-9341afbfca99 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:30.369754 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2918be6d-70c9-417d-8061-9341afbfca99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:30.426670 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1307/2613] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:30.438000 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31728ff4-f606-49a9-a92b-e9c20d64995b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:30.439602 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-31728ff4-f606-49a9-a92b-e9c20d64995b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:30.447904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918be6d-70c9-417d-8061-9341afbfca99 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:30.454217 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf52d94-6a11-4ac7-bb23-659b4b4ec168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:30.454957 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf52d94-6a11-4ac7-bb23-659b4b4ec168 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:30.455243 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf52d94-6a11-4ac7-bb23-659b4b4ec168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:30.455514 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf52d94-6a11-4ac7-bb23-659b4b4ec168 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:30.456423 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf52d94-6a11-4ac7-bb23-659b4b4ec168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:30.457865 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf52d94-6a11-4ac7-bb23-659b4b4ec168 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:30.459225 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:30.471973 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:30.480427 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1307/2614] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:30 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:16:30.482176 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf52d94-6a11-4ac7-bb23-659b4b4ec168 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:30.494499 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf52d94-6a11-4ac7-bb23-659b4b4ec168 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:30.499863 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151975dc-19f0-4ceb-a974-ad120db6480f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:30.500520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151975dc-19f0-4ceb-a974-ad120db6480f admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:30.500795 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151975dc-19f0-4ceb-a974-ad120db6480f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:30.501068 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151975dc-19f0-4ceb-a974-ad120db6480f admin admin] PATH_INFO: `/v3/projects/0de7d515d7564e82944640dc066acabf/users/56a7b2ddd7b24a83adc885754dea95c1/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:30.503086 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151975dc-19f0-4ceb-a974-ad120db6480f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de7d515d7564e82944640dc066acabf, user_id=56a7b2ddd7b24a83adc885754dea95c1, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:30.505800 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151975dc-19f0-4ceb-a974-ad120db6480f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:30.518794 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31728ff4-f606-49a9-a92b-e9c20d64995b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:30.523246 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1308/2615] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:30 2022] PUT /identity/v3/projects/0de7d515d7564e82944640dc066acabf/users/56a7b2ddd7b24a83adc885754dea95c1/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:16:30.534001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919eff8b-aaec-4f36-abd2-e7bc44eb086b None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:30.534364 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919eff8b-aaec-4f36-abd2-e7bc44eb086b None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:30.534573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919eff8b-aaec-4f36-abd2-e7bc44eb086b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:30.567672 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-919eff8b-aaec-4f36-abd2-e7bc44eb086b None None] MFA Rules not processed for user `56a7b2ddd7b24a83adc885754dea95c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:30.569191 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-919eff8b-aaec-4f36-abd2-e7bc44eb086b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:30.576094 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316e2b6-f711-407f-8d11-3f1ec708403f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:30.576800 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316e2b6-f711-407f-8d11-3f1ec708403f service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:30.577211 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316e2b6-f711-407f-8d11-3f1ec708403f service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:30.577561 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316e2b6-f711-407f-8d11-3f1ec708403f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:30.647637 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316e2b6-f711-407f-8d11-3f1ec708403f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:30.649276 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316e2b6-f711-407f-8d11-3f1ec708403f service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:30.655682 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1309/2616] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:30.660890 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1308/2617] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:16:30.669377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919eff8b-aaec-4f36-abd2-e7bc44eb086b None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:30.672687 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-919eff8b-aaec-4f36-abd2-e7bc44eb086b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:30.757905 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919eff8b-aaec-4f36-abd2-e7bc44eb086b None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:30.762216 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf83b7f-33e9-4f1b-aacf-96e1cc6b5b5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:30.763770 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf83b7f-33e9-4f1b-aacf-96e1cc6b5b5c service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:30.764120 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf83b7f-33e9-4f1b-aacf-96e1cc6b5b5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:30.764525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf83b7f-33e9-4f1b-aacf-96e1cc6b5b5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:30.766984 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0bf83b7f-33e9-4f1b-aacf-96e1cc6b5b5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:30.837451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf83b7f-33e9-4f1b-aacf-96e1cc6b5b5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:30.838898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf83b7f-33e9-4f1b-aacf-96e1cc6b5b5c service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:30.849774 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1310/2618] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:16:30.948828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316e2b6-f711-407f-8d11-3f1ec708403f service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:30.950290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f316e2b6-f711-407f-8d11-3f1ec708403f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:30.963996 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316e2b6-f711-407f-8d11-3f1ec708403f service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:30.970089 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac75de-d23d-4855-8889-b91a60525cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:30.970805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac75de-d23d-4855-8889-b91a60525cd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:30.971007 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac75de-d23d-4855-8889-b91a60525cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:30.971285 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac75de-d23d-4855-8889-b91a60525cd4 admin admin] PATH_INFO: `/v3/projects/1d348cdc67014251875439ca27223e80` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:30.972572 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ac75de-d23d-4855-8889-b91a60525cd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d348cdc67014251875439ca27223e80)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:30.974388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ac75de-d23d-4855-8889-b91a60525cd4 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:31.027353 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d7ac75de-d23d-4855-8889-b91a60525cd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d348cdc67014251875439ca27223e80', 'request_id': 'req-d7ac75de-d23d-4855-8889-b91a60525cd4'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:31.036186 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d7ac75de-d23d-4855-8889-b91a60525cd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d348cdc67014251875439ca27223e80', 'request_id': 'req-d7ac75de-d23d-4855-8889-b91a60525cd4'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:31.042415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d7ac75de-d23d-4855-8889-b91a60525cd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d348cdc67014251875439ca27223e80', 'request_id': 'req-d7ac75de-d23d-4855-8889-b91a60525cd4'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:31.054905 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d7ac75de-d23d-4855-8889-b91a60525cd4 admin admin] The token cache is being invalidate because project 1d348cdc67014251875439ca27223e80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:31.055199 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d7ac75de-d23d-4855-8889-b91a60525cd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:31.055881 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d7ac75de-d23d-4855-8889-b91a60525cd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:31.057366 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1309/2619] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:30 2022] DELETE /identity/v3/projects/1d348cdc67014251875439ca27223e80 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:31.061624 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf83b7f-33e9-4f1b-aacf-96e1cc6b5b5c service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:31.072507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf83b7f-33e9-4f1b-aacf-96e1cc6b5b5c service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:31.093154 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:31.140904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023dfc29-6f28-4da1-b723-912c3d745725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:31.141594 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023dfc29-6f28-4da1-b723-912c3d745725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:31.141983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023dfc29-6f28-4da1-b723-912c3d745725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:31.142211 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023dfc29-6f28-4da1-b723-912c3d745725 admin admin] PATH_INFO: `/v3/users/b12daa55c1a34e8189d7cef265efa86d` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:31.143779 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023dfc29-6f28-4da1-b723-912c3d745725 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b12daa55c1a34e8189d7cef265efa86d)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:31.145863 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023dfc29-6f28-4da1-b723-912c3d745725 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:31.201162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-023dfc29-6f28-4da1-b723-912c3d745725 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b12daa55c1a34e8189d7cef265efa86d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:31.201815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-023dfc29-6f28-4da1-b723-912c3d745725 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b12daa55c1a34e8189d7cef265efa86d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:31.211231 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-023dfc29-6f28-4da1-b723-912c3d745725 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b12daa55c1a34e8189d7cef265efa86d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:31.211858 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-023dfc29-6f28-4da1-b723-912c3d745725 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b12daa55c1a34e8189d7cef265efa86d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:31.220673 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-023dfc29-6f28-4da1-b723-912c3d745725 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b12daa55c1a34e8189d7cef265efa86d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:31.238478 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1311/2620] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:31 2022] DELETE /identity/v3/users/b12daa55c1a34e8189d7cef265efa86d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:31.512799 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac75de-d23d-4855-8889-b91a60525cd4 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:31.514355 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d7ac75de-d23d-4855-8889-b91a60525cd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:31.528134 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac75de-d23d-4855-8889-b91a60525cd4 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:31.537500 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:31.589607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85be434-de83-4284-b9ca-b7145fce031f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:31.590320 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85be434-de83-4284-b9ca-b7145fce031f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:31.590657 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85be434-de83-4284-b9ca-b7145fce031f admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:31.590969 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85be434-de83-4284-b9ca-b7145fce031f admin admin] PATH_INFO: `/v3/projects/fc1807a0636c45b0b7ef235eb3cf4f3d` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:31.592429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85be434-de83-4284-b9ca-b7145fce031f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc1807a0636c45b0b7ef235eb3cf4f3d)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:31.594800 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85be434-de83-4284-b9ca-b7145fce031f admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:31.641524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d85be434-de83-4284-b9ca-b7145fce031f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc1807a0636c45b0b7ef235eb3cf4f3d', 'request_id': 'req-d85be434-de83-4284-b9ca-b7145fce031f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:31.648891 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d85be434-de83-4284-b9ca-b7145fce031f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc1807a0636c45b0b7ef235eb3cf4f3d', 'request_id': 'req-d85be434-de83-4284-b9ca-b7145fce031f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:31.654723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d85be434-de83-4284-b9ca-b7145fce031f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc1807a0636c45b0b7ef235eb3cf4f3d', 'request_id': 'req-d85be434-de83-4284-b9ca-b7145fce031f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:31.666020 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d85be434-de83-4284-b9ca-b7145fce031f admin admin] The token cache is being invalidate because project fc1807a0636c45b0b7ef235eb3cf4f3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:31.666421 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d85be434-de83-4284-b9ca-b7145fce031f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:31.666941 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d85be434-de83-4284-b9ca-b7145fce031f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:31.668457 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1310/2621] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:31 2022] DELETE /identity/v3/projects/fc1807a0636c45b0b7ef235eb3cf4f3d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:31.673571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d802a6-c2ed-4e29-981c-2a19c40f05aa None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:31.674048 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d802a6-c2ed-4e29-981c-2a19c40f05aa None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:31.674192 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d802a6-c2ed-4e29-981c-2a19c40f05aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:31.695156 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-50d802a6-c2ed-4e29-981c-2a19c40f05aa None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:31.696421 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-50d802a6-c2ed-4e29-981c-2a19c40f05aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:31.755718 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1312/2622] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:31.763871 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e924ecfc-0c68-4c9f-80fa-b139a501477e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:31.764168 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e924ecfc-0c68-4c9f-80fa-b139a501477e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:31.764507 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e924ecfc-0c68-4c9f-80fa-b139a501477e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:31.791002 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e924ecfc-0c68-4c9f-80fa-b139a501477e None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:31.792274 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e924ecfc-0c68-4c9f-80fa-b139a501477e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:31.800826 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1311/2623] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:31.805166 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d802a6-c2ed-4e29-981c-2a19c40f05aa None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:31.816271 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d802a6-c2ed-4e29-981c-2a19c40f05aa None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:31.821730 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea7649-9b34-497e-8e8f-a3e8f835c287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:31.822419 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea7649-9b34-497e-8e8f-a3e8f835c287 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:31.822762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea7649-9b34-497e-8e8f-a3e8f835c287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:31.823095 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea7649-9b34-497e-8e8f-a3e8f835c287 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:31.823794 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ea7649-9b34-497e-8e8f-a3e8f835c287 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:16:31.824099 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ea7649-9b34-497e-8e8f-a3e8f835c287 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:31.825711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ea7649-9b34-497e-8e8f-a3e8f835c287 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:31.838509 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1313/2624] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:16:31.846580 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e924ecfc-0c68-4c9f-80fa-b139a501477e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:31.852861 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e924ecfc-0c68-4c9f-80fa-b139a501477e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:31.857537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40c4f7-bdd1-419b-acb7-e3ade217c5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:31.858132 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40c4f7-bdd1-419b-acb7-e3ade217c5c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:31.859107 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40c4f7-bdd1-419b-acb7-e3ade217c5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:31.859760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40c4f7-bdd1-419b-acb7-e3ade217c5c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:31.860273 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40c4f7-bdd1-419b-acb7-e3ade217c5c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:31.862168 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40c4f7-bdd1-419b-acb7-e3ade217c5c4 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:31.894679 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1312/2625] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:31 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:31.896098 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea7649-9b34-497e-8e8f-a3e8f835c287 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:31.907383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea7649-9b34-497e-8e8f-a3e8f835c287 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:31.934916 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:31.970493 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2748bcc3-80fb-4cd9-a197-40af16bb4c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:31.971152 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748bcc3-80fb-4cd9-a197-40af16bb4c86 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:31.971438 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748bcc3-80fb-4cd9-a197-40af16bb4c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:31.971755 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2748bcc3-80fb-4cd9-a197-40af16bb4c86 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:31.972392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2748bcc3-80fb-4cd9-a197-40af16bb4c86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:31.974356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2748bcc3-80fb-4cd9-a197-40af16bb4c86 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:31.997163 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1314/2626] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:31 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:32.001397 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40c4f7-bdd1-419b-acb7-e3ade217c5c4 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:32.012378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40c4f7-bdd1-419b-acb7-e3ade217c5c4 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:32.018232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddcf34-b260-4741-b136-a98c734caee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:32.018963 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddcf34-b260-4741-b136-a98c734caee7 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:32.019249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddcf34-b260-4741-b136-a98c734caee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:32.019580 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddcf34-b260-4741-b136-a98c734caee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:32.020232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fddcf34-b260-4741-b136-a98c734caee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:32.021871 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fddcf34-b260-4741-b136-a98c734caee7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:32.033667 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1313/2627] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:32.038048 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2748bcc3-80fb-4cd9-a197-40af16bb4c86 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:32.049426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2748bcc3-80fb-4cd9-a197-40af16bb4c86 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:32.054434 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76a19e-d292-47f9-8f9d-f34dcf7590b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:32.055052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76a19e-d292-47f9-8f9d-f34dcf7590b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:32.055273 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76a19e-d292-47f9-8f9d-f34dcf7590b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:32.055564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76a19e-d292-47f9-8f9d-f34dcf7590b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:32.056232 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76a19e-d292-47f9-8f9d-f34dcf7590b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:32.057917 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76a19e-d292-47f9-8f9d-f34dcf7590b7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:32.069273 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1315/2628] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:32.073540 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddcf34-b260-4741-b136-a98c734caee7 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:32.086393 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddcf34-b260-4741-b136-a98c734caee7 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:32.091387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dbac32-bf80-4f5c-a819-1e0996f8ecfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:32.092131 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbac32-bf80-4f5c-a819-1e0996f8ecfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:32.092508 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbac32-bf80-4f5c-a819-1e0996f8ecfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:32.093756 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbac32-bf80-4f5c-a819-1e0996f8ecfc admin admin] PATH_INFO: `/v3/projects/267fca4ef0044788a0aed0b312dbfbff/users/91de53e7eb664cdd95700e9a182285a3/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:32.111563 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dbac32-bf80-4f5c-a819-1e0996f8ecfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=267fca4ef0044788a0aed0b312dbfbff, user_id=91de53e7eb664cdd95700e9a182285a3, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:32.114262 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dbac32-bf80-4f5c-a819-1e0996f8ecfc admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:32.130923 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1314/2629] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:32 2022] PUT /identity/v3/projects/267fca4ef0044788a0aed0b312dbfbff/users/91de53e7eb664cdd95700e9a182285a3/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:32.143937 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb412b-c3b5-4421-9d96-b2f723edd11d None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:32.144576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb412b-c3b5-4421-9d96-b2f723edd11d None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:32.145023 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb412b-c3b5-4421-9d96-b2f723edd11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:32.174640 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-2cfb412b-c3b5-4421-9d96-b2f723edd11d None None] MFA Rules not processed for user `91de53e7eb664cdd95700e9a182285a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:32.175973 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2cfb412b-c3b5-4421-9d96-b2f723edd11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:32.244631 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1316/2630] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:32.251834 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dbac32-bf80-4f5c-a819-1e0996f8ecfc admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:32.296307 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:32.320018 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dbac32-bf80-4f5c-a819-1e0996f8ecfc admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:32.324788 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068dad5-f91f-4275-9057-39119937bfd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:32.325400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068dad5-f91f-4275-9057-39119937bfd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:32.325686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068dad5-f91f-4275-9057-39119937bfd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:32.325966 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068dad5-f91f-4275-9057-39119937bfd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:32.326605 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8068dad5-f91f-4275-9057-39119937bfd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:32.329252 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8068dad5-f91f-4275-9057-39119937bfd6 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:32.357030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1315/2631] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:32 2022] POST /identity/v3/projects => generated 363 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:32.364608 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb412b-c3b5-4421-9d96-b2f723edd11d None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:32.376362 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb412b-c3b5-4421-9d96-b2f723edd11d None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:32.397471 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068dad5-f91f-4275-9057-39119937bfd6 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:32.399197 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-8068dad5-f91f-4275-9057-39119937bfd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:32.437967 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a4789-02c0-4b8d-8286-1f62b571b877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:32.438576 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a4789-02c0-4b8d-8286-1f62b571b877 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:32.438852 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a4789-02c0-4b8d-8286-1f62b571b877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:32.439125 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a4789-02c0-4b8d-8286-1f62b571b877 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:32.439776 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a4789-02c0-4b8d-8286-1f62b571b877 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:32.441732 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a4789-02c0-4b8d-8286-1f62b571b877 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:32.458767 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068dad5-f91f-4275-9057-39119937bfd6 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:32.462497 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ef9b9-ca90-4782-a40f-c7196dc8ee37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:32.463087 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1317/2632] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:32 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:32.463225 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ef9b9-ca90-4782-a40f-c7196dc8ee37 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:32.466402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ef9b9-ca90-4782-a40f-c7196dc8ee37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:32.466698 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ef9b9-ca90-4782-a40f-c7196dc8ee37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:32.467007 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a4789-02c0-4b8d-8286-1f62b571b877 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:32.472758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e19ef9b9-ca90-4782-a40f-c7196dc8ee37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:32.477231 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a4789-02c0-4b8d-8286-1f62b571b877 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:32.482261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f8185-6e85-4474-8eb6-cdc5bb43972f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:32.482999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f8185-6e85-4474-8eb6-cdc5bb43972f admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:32.483377 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f8185-6e85-4474-8eb6-cdc5bb43972f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:32.483832 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f8185-6e85-4474-8eb6-cdc5bb43972f admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:32.484495 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f8185-6e85-4474-8eb6-cdc5bb43972f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:32.486082 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f8185-6e85-4474-8eb6-cdc5bb43972f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:32.499761 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1318/2633] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:16:32.500927 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f8185-6e85-4474-8eb6-cdc5bb43972f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:32.511061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f8185-6e85-4474-8eb6-cdc5bb43972f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:32.516511 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aee33d-d151-4df4-83b6-3224d57fbd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:32.517149 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aee33d-d151-4df4-83b6-3224d57fbd2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:32.517426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aee33d-d151-4df4-83b6-3224d57fbd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:32.517698 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aee33d-d151-4df4-83b6-3224d57fbd2b admin admin] PATH_INFO: `/v3/projects/8945d7c3388640c08a69189399eb5ce2/users/21696ae854e542f2963ca98675d434ae/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:32.531228 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:32.533174 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ef9b9-ca90-4782-a40f-c7196dc8ee37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:32.534669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ef9b9-ca90-4782-a40f-c7196dc8ee37 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:32.534823 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aee33d-d151-4df4-83b6-3224d57fbd2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8945d7c3388640c08a69189399eb5ce2, user_id=21696ae854e542f2963ca98675d434ae, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:32.537774 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aee33d-d151-4df4-83b6-3224d57fbd2b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:32.544156 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1316/2634] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:16:32.551762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1319/2635] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:32 2022] PUT /identity/v3/projects/8945d7c3388640c08a69189399eb5ce2/users/21696ae854e542f2963ca98675d434ae/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:16:32.555939 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ef9b9-ca90-4782-a40f-c7196dc8ee37 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:32.567159 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ef9b9-ca90-4782-a40f-c7196dc8ee37 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:32.625262 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a55f6-a64f-40b8-9dea-b21277de4376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:32.625803 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a55f6-a64f-40b8-9dea-b21277de4376 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:32.626236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a55f6-a64f-40b8-9dea-b21277de4376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:32.626533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a55f6-a64f-40b8-9dea-b21277de4376 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:32.627198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a55f6-a64f-40b8-9dea-b21277de4376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:32.628829 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a55f6-a64f-40b8-9dea-b21277de4376 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:32.639333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1317/2636] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:32.644427 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aee33d-d151-4df4-83b6-3224d57fbd2b admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:32.655648 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aee33d-d151-4df4-83b6-3224d57fbd2b admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:32.660662 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40199d-6622-46e8-a9cd-763195f19d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:32.661337 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40199d-6622-46e8-a9cd-763195f19d79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:32.661624 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40199d-6622-46e8-a9cd-763195f19d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:32.661905 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40199d-6622-46e8-a9cd-763195f19d79 admin admin] PATH_INFO: `/v3/projects/8945d7c3388640c08a69189399eb5ce2/users/21696ae854e542f2963ca98675d434ae/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:32.663925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40199d-6622-46e8-a9cd-763195f19d79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8945d7c3388640c08a69189399eb5ce2, user_id=21696ae854e542f2963ca98675d434ae, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:32.666707 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40199d-6622-46e8-a9cd-763195f19d79 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:32.677947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1320/2637] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:32 2022] PUT /identity/v3/projects/8945d7c3388640c08a69189399eb5ce2/users/21696ae854e542f2963ca98675d434ae/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:32.689324 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f7c61-f80d-4864-bbec-82ad2c21196f None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:32.689619 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f7c61-f80d-4864-bbec-82ad2c21196f None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:32.689900 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f7c61-f80d-4864-bbec-82ad2c21196f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:32.716522 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:32.720063 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-843f7c61-f80d-4864-bbec-82ad2c21196f None None] MFA Rules not processed for user `21696ae854e542f2963ca98675d434ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:32.721957 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-843f7c61-f80d-4864-bbec-82ad2c21196f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:32.794536 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1318/2638] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:32 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:32.806800 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40199d-6622-46e8-a9cd-763195f19d79 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:32.808630 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4c40199d-6622-46e8-a9cd-763195f19d79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:32.866693 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40199d-6622-46e8-a9cd-763195f19d79 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:32.870299 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad84a92-e66c-4273-81ba-59b81c32df36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:32.870898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad84a92-e66c-4273-81ba-59b81c32df36 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:32.871176 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad84a92-e66c-4273-81ba-59b81c32df36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:32.871451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad84a92-e66c-4273-81ba-59b81c32df36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:32.881846 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad84a92-e66c-4273-81ba-59b81c32df36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:32.883466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad84a92-e66c-4273-81ba-59b81c32df36 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:32.893084 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1321/2639] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:32 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:32.967134 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f7c61-f80d-4864-bbec-82ad2c21196f None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:32.976797 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f7c61-f80d-4864-bbec-82ad2c21196f None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:32.980347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea52701-5d92-4605-aa8f-ac7724a03c3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:32.980943 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea52701-5d92-4605-aa8f-ac7724a03c3a service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:32.981222 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea52701-5d92-4605-aa8f-ac7724a03c3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:32.981494 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea52701-5d92-4605-aa8f-ac7724a03c3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:33.039389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea52701-5d92-4605-aa8f-ac7724a03c3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:33.040970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea52701-5d92-4605-aa8f-ac7724a03c3a service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:33.051263 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1319/2640] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:33.320841 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad84a92-e66c-4273-81ba-59b81c32df36 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:33.330006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad84a92-e66c-4273-81ba-59b81c32df36 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:33.333879 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239ba6d-0783-4abb-9288-57d39df82d16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:33.334480 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239ba6d-0783-4abb-9288-57d39df82d16 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:33.334759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239ba6d-0783-4abb-9288-57d39df82d16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:33.335029 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239ba6d-0783-4abb-9288-57d39df82d16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:33.336777 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1239ba6d-0783-4abb-9288-57d39df82d16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:33.407634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239ba6d-0783-4abb-9288-57d39df82d16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:33.409323 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239ba6d-0783-4abb-9288-57d39df82d16 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:33.419176 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1322/2641] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:33 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:34.721686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea52701-5d92-4605-aa8f-ac7724a03c3a service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:34.730550 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea52701-5d92-4605-aa8f-ac7724a03c3a service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:34.734759 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1825335-1d6f-4c25-91cf-fcf2f6ff713a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:34.735541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1825335-1d6f-4c25-91cf-fcf2f6ff713a service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:34.736004 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1825335-1d6f-4c25-91cf-fcf2f6ff713a service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:34.736430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1825335-1d6f-4c25-91cf-fcf2f6ff713a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:34.738662 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c1825335-1d6f-4c25-91cf-fcf2f6ff713a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:34.809616 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1825335-1d6f-4c25-91cf-fcf2f6ff713a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:34.811183 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1825335-1d6f-4c25-91cf-fcf2f6ff713a service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:34.821425 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1320/2642] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:35.209901 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239ba6d-0783-4abb-9288-57d39df82d16 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:35.275619 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239ba6d-0783-4abb-9288-57d39df82d16 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:35.280339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34aa37-bd56-460f-a5bd-b22ecf578cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:35.280973 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34aa37-bd56-460f-a5bd-b22ecf578cd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:35.281254 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34aa37-bd56-460f-a5bd-b22ecf578cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:35.281529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34aa37-bd56-460f-a5bd-b22ecf578cd9 admin admin] PATH_INFO: `/v3/users/91de53e7eb664cdd95700e9a182285a3` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:35.282649 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34aa37-bd56-460f-a5bd-b22ecf578cd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91de53e7eb664cdd95700e9a182285a3)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:35.284625 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34aa37-bd56-460f-a5bd-b22ecf578cd9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:35.302151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:35.332436 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eb34aa37-bd56-460f-a5bd-b22ecf578cd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91de53e7eb664cdd95700e9a182285a3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:35.333078 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eb34aa37-bd56-460f-a5bd-b22ecf578cd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91de53e7eb664cdd95700e9a182285a3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:35.339893 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eb34aa37-bd56-460f-a5bd-b22ecf578cd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91de53e7eb664cdd95700e9a182285a3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:35.340600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eb34aa37-bd56-460f-a5bd-b22ecf578cd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91de53e7eb664cdd95700e9a182285a3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:35.348458 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-eb34aa37-bd56-460f-a5bd-b22ecf578cd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91de53e7eb664cdd95700e9a182285a3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:35.366406 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1323/2643] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:35 2022] DELETE /identity/v3/users/91de53e7eb664cdd95700e9a182285a3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:35.370998 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf164f-85ab-4dae-a285-fe5d0b0d12a6 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:35.371295 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf164f-85ab-4dae-a285-fe5d0b0d12a6 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:35.371582 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf164f-85ab-4dae-a285-fe5d0b0d12a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:35.390908 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-6ebf164f-85ab-4dae-a285-fe5d0b0d12a6 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:35.392144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6ebf164f-85ab-4dae-a285-fe5d0b0d12a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:35.417521 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:35.450869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1321/2644] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:35.461216 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34aa37-bd56-460f-a5bd-b22ecf578cd9 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:35.462646 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-eb34aa37-bd56-460f-a5bd-b22ecf578cd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:35.521119 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34aa37-bd56-460f-a5bd-b22ecf578cd9 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:35.524951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b1e93-2fdd-4c1e-8ba2-9f55518c0483 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:35.525563 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b1e93-2fdd-4c1e-8ba2-9f55518c0483 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:35.525842 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b1e93-2fdd-4c1e-8ba2-9f55518c0483 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:35.526116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b1e93-2fdd-4c1e-8ba2-9f55518c0483 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:35.536189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b1e93-2fdd-4c1e-8ba2-9f55518c0483 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:35.537628 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b1e93-2fdd-4c1e-8ba2-9f55518c0483 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:35.547935 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1324/2645] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:35.813553 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf164f-85ab-4dae-a285-fe5d0b0d12a6 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:35.815122 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6ebf164f-85ab-4dae-a285-fe5d0b0d12a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:35.829053 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf164f-85ab-4dae-a285-fe5d0b0d12a6 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:35.834582 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a6b75-6d09-494a-9404-b53e180ead4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:35.835269 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a6b75-6d09-494a-9404-b53e180ead4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:35.835591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a6b75-6d09-494a-9404-b53e180ead4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:35.835860 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a6b75-6d09-494a-9404-b53e180ead4e admin admin] PATH_INFO: `/v3/projects/267fca4ef0044788a0aed0b312dbfbff` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:35.837423 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a6b75-6d09-494a-9404-b53e180ead4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=267fca4ef0044788a0aed0b312dbfbff)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:35.840089 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a6b75-6d09-494a-9404-b53e180ead4e admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:35.899765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-981a6b75-6d09-494a-9404-b53e180ead4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '267fca4ef0044788a0aed0b312dbfbff', 'request_id': 'req-981a6b75-6d09-494a-9404-b53e180ead4e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:35.910618 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-981a6b75-6d09-494a-9404-b53e180ead4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '267fca4ef0044788a0aed0b312dbfbff', 'request_id': 'req-981a6b75-6d09-494a-9404-b53e180ead4e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:35.924865 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-981a6b75-6d09-494a-9404-b53e180ead4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '267fca4ef0044788a0aed0b312dbfbff', 'request_id': 'req-981a6b75-6d09-494a-9404-b53e180ead4e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:35.948713 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-981a6b75-6d09-494a-9404-b53e180ead4e admin admin] The token cache is being invalidate because project 267fca4ef0044788a0aed0b312dbfbff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:35.949030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-981a6b75-6d09-494a-9404-b53e180ead4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:35.949689 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-981a6b75-6d09-494a-9404-b53e180ead4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:35.951172 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1322/2646] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:35 2022] DELETE /identity/v3/projects/267fca4ef0044788a0aed0b312dbfbff => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:35.955910 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b1e93-2fdd-4c1e-8ba2-9f55518c0483 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:35.965524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b1e93-2fdd-4c1e-8ba2-9f55518c0483 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:36.024125 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92163894-4dfb-4077-adfb-8099c59925f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:36.024754 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92163894-4dfb-4077-adfb-8099c59925f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:36.025052 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92163894-4dfb-4077-adfb-8099c59925f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:36.025303 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92163894-4dfb-4077-adfb-8099c59925f4 admin admin] PATH_INFO: `/v3/users/21696ae854e542f2963ca98675d434ae` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:36.026398 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92163894-4dfb-4077-adfb-8099c59925f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21696ae854e542f2963ca98675d434ae)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:36.028336 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92163894-4dfb-4077-adfb-8099c59925f4 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:36.068601 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:36.075612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-92163894-4dfb-4077-adfb-8099c59925f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21696ae854e542f2963ca98675d434ae'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:36.076698 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-92163894-4dfb-4077-adfb-8099c59925f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21696ae854e542f2963ca98675d434ae'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:36.084123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-92163894-4dfb-4077-adfb-8099c59925f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21696ae854e542f2963ca98675d434ae'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:36.084757 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-92163894-4dfb-4077-adfb-8099c59925f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21696ae854e542f2963ca98675d434ae'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:36.091892 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-92163894-4dfb-4077-adfb-8099c59925f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21696ae854e542f2963ca98675d434ae'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:36.109966 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1325/2647] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:35 2022] DELETE /identity/v3/users/21696ae854e542f2963ca98675d434ae => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:36.408909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a6b75-6d09-494a-9404-b53e180ead4e admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:36.411114 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-981a6b75-6d09-494a-9404-b53e180ead4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:36.426118 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a6b75-6d09-494a-9404-b53e180ead4e admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:36.487537 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9386fb-9b7d-4899-9c49-d684b88c5a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:36.488198 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9386fb-9b7d-4899-9c49-d684b88c5a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:36.488478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9386fb-9b7d-4899-9c49-d684b88c5a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:36.488749 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9386fb-9b7d-4899-9c49-d684b88c5a8e admin admin] PATH_INFO: `/v3/projects/8945d7c3388640c08a69189399eb5ce2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:36.489874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9386fb-9b7d-4899-9c49-d684b88c5a8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8945d7c3388640c08a69189399eb5ce2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:36.491814 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9386fb-9b7d-4899-9c49-d684b88c5a8e admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:36.538624 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-cd9386fb-9b7d-4899-9c49-d684b88c5a8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8945d7c3388640c08a69189399eb5ce2', 'request_id': 'req-cd9386fb-9b7d-4899-9c49-d684b88c5a8e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:36.545724 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-cd9386fb-9b7d-4899-9c49-d684b88c5a8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8945d7c3388640c08a69189399eb5ce2', 'request_id': 'req-cd9386fb-9b7d-4899-9c49-d684b88c5a8e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:36.547092 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:36.553179 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-cd9386fb-9b7d-4899-9c49-d684b88c5a8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8945d7c3388640c08a69189399eb5ce2', 'request_id': 'req-cd9386fb-9b7d-4899-9c49-d684b88c5a8e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:36.564818 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-cd9386fb-9b7d-4899-9c49-d684b88c5a8e admin admin] The token cache is being invalidate because project 8945d7c3388640c08a69189399eb5ce2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:36.565107 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-cd9386fb-9b7d-4899-9c49-d684b88c5a8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:36.565775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-cd9386fb-9b7d-4899-9c49-d684b88c5a8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:36.567263 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1323/2648] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:36 2022] DELETE /identity/v3/projects/8945d7c3388640c08a69189399eb5ce2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:36.572291 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa18c1-8d14-48e9-a3e9-c788287b11f6 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:36.572761 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa18c1-8d14-48e9-a3e9-c788287b11f6 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:36.572862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa18c1-8d14-48e9-a3e9-c788287b11f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:36.593281 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-fefa18c1-8d14-48e9-a3e9-c788287b11f6 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:36.594475 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-fefa18c1-8d14-48e9-a3e9-c788287b11f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:36.653047 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1326/2649] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:36.662805 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7dd8e-56de-4b89-b610-4e7d0469532e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:36.663099 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7dd8e-56de-4b89-b610-4e7d0469532e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:36.663388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc7dd8e-56de-4b89-b610-4e7d0469532e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:36.683139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-4cc7dd8e-56de-4b89-b610-4e7d0469532e None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:36.684381 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4cc7dd8e-56de-4b89-b610-4e7d0469532e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:36.693020 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1324/2650] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:36.697321 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa18c1-8d14-48e9-a3e9-c788287b11f6 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:36.707856 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa18c1-8d14-48e9-a3e9-c788287b11f6 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:36.712456 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba2139-ad7c-42e1-b764-8bdb9b84b3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:36.713061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba2139-ad7c-42e1-b764-8bdb9b84b3ba admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:36.713344 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba2139-ad7c-42e1-b764-8bdb9b84b3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:36.713620 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba2139-ad7c-42e1-b764-8bdb9b84b3ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:36.714275 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba2139-ad7c-42e1-b764-8bdb9b84b3ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:16:36.714584 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba2139-ad7c-42e1-b764-8bdb9b84b3ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:36.716226 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba2139-ad7c-42e1-b764-8bdb9b84b3ba admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:36.728401 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1327/2651] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:16:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:16:36.732763 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7dd8e-56de-4b89-b610-4e7d0469532e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:36.742382 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc7dd8e-56de-4b89-b610-4e7d0469532e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:36.747345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756b0c3-346b-44b7-9fbe-4294d9e88cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:36.747989 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756b0c3-346b-44b7-9fbe-4294d9e88cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:36.748275 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756b0c3-346b-44b7-9fbe-4294d9e88cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:36.748556 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756b0c3-346b-44b7-9fbe-4294d9e88cc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:36.749518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756b0c3-346b-44b7-9fbe-4294d9e88cc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:36.751512 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756b0c3-346b-44b7-9fbe-4294d9e88cc7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:36.778566 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1325/2652] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:36 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:36.783423 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba2139-ad7c-42e1-b764-8bdb9b84b3ba admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:36.793447 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba2139-ad7c-42e1-b764-8bdb9b84b3ba admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:36.851079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2bb77-e457-4b44-91e2-31300b891325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:36.851733 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2bb77-e457-4b44-91e2-31300b891325 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:36.852042 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2bb77-e457-4b44-91e2-31300b891325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:36.852314 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2bb77-e457-4b44-91e2-31300b891325 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:36.852947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2bb77-e457-4b44-91e2-31300b891325 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:36.855047 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2bb77-e457-4b44-91e2-31300b891325 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:36.876707 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1328/2653] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:36 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:36.882146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756b0c3-346b-44b7-9fbe-4294d9e88cc7 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:36.893781 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756b0c3-346b-44b7-9fbe-4294d9e88cc7 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:36.898311 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfc99b-e929-4cc3-8eea-ac7df8889f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:36.898904 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfc99b-e929-4cc3-8eea-ac7df8889f93 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:36.899179 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfc99b-e929-4cc3-8eea-ac7df8889f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:36.899449 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfc99b-e929-4cc3-8eea-ac7df8889f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:36.900133 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfc99b-e929-4cc3-8eea-ac7df8889f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:36.901715 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfc99b-e929-4cc3-8eea-ac7df8889f93 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:36.912408 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1326/2654] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:36.916375 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2bb77-e457-4b44-91e2-31300b891325 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:36.926641 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2bb77-e457-4b44-91e2-31300b891325 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:36.931418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a7747-3740-48a4-9460-bffd884c5222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:36.932053 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a7747-3740-48a4-9460-bffd884c5222 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:36.932339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a7747-3740-48a4-9460-bffd884c5222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:36.932622 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a7747-3740-48a4-9460-bffd884c5222 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:36.933268 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855a7747-3740-48a4-9460-bffd884c5222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:36.934858 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855a7747-3740-48a4-9460-bffd884c5222 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:36.946350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1329/2655] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:36.950673 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfc99b-e929-4cc3-8eea-ac7df8889f93 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:36.967834 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfc99b-e929-4cc3-8eea-ac7df8889f93 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:36.972849 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85388fac-7f22-4612-a85c-8e1481460993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:36.973511 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85388fac-7f22-4612-a85c-8e1481460993 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:36.973791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85388fac-7f22-4612-a85c-8e1481460993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:36.974073 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85388fac-7f22-4612-a85c-8e1481460993 admin admin] PATH_INFO: `/v3/projects/0f7287858f5048ad9d1d9565037cfb62/users/7886cb05d5b14a628aa5e053035af818/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:36.991517 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85388fac-7f22-4612-a85c-8e1481460993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f7287858f5048ad9d1d9565037cfb62, user_id=7886cb05d5b14a628aa5e053035af818, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:36.994261 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85388fac-7f22-4612-a85c-8e1481460993 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:37.011200 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1327/2656] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:36 2022] PUT /identity/v3/projects/0f7287858f5048ad9d1d9565037cfb62/users/7886cb05d5b14a628aa5e053035af818/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:37.023190 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca917cc-69a7-4f89-8d1e-8c8c92dc71e8 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:37.023506 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca917cc-69a7-4f89-8d1e-8c8c92dc71e8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:37.023863 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca917cc-69a7-4f89-8d1e-8c8c92dc71e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:37.054524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-cca917cc-69a7-4f89-8d1e-8c8c92dc71e8 None None] MFA Rules not processed for user `7886cb05d5b14a628aa5e053035af818`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:37.055924 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cca917cc-69a7-4f89-8d1e-8c8c92dc71e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:37.067100 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:37.130779 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1330/2657] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:37.135654 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85388fac-7f22-4612-a85c-8e1481460993 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:37.205065 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85388fac-7f22-4612-a85c-8e1481460993 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:37.210419 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf25374-1c8c-4eb8-93f7-c0a5a3a04d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:37.211024 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf25374-1c8c-4eb8-93f7-c0a5a3a04d04 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:37.211305 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf25374-1c8c-4eb8-93f7-c0a5a3a04d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:37.211607 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf25374-1c8c-4eb8-93f7-c0a5a3a04d04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:37.212243 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf25374-1c8c-4eb8-93f7-c0a5a3a04d04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:37.214174 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf25374-1c8c-4eb8-93f7-c0a5a3a04d04 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:37.247105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1328/2658] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:37 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:37.249639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca917cc-69a7-4f89-8d1e-8c8c92dc71e8 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:37.262977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca917cc-69a7-4f89-8d1e-8c8c92dc71e8 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:37.324226 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b589e87-2fcc-4bb1-8b16-e1ee5fcbb8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:37.324851 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b589e87-2fcc-4bb1-8b16-e1ee5fcbb8e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:37.325137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b589e87-2fcc-4bb1-8b16-e1ee5fcbb8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:37.325413 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b589e87-2fcc-4bb1-8b16-e1ee5fcbb8e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:37.326060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b589e87-2fcc-4bb1-8b16-e1ee5fcbb8e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:37.328223 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b589e87-2fcc-4bb1-8b16-e1ee5fcbb8e4 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:37.355466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1331/2659] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:37 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:16:37.357017 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf25374-1c8c-4eb8-93f7-c0a5a3a04d04 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:37.370264 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf25374-1c8c-4eb8-93f7-c0a5a3a04d04 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:37.375919 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5a2b4-3c7f-4374-a108-868c110c64fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:37.376527 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5a2b4-3c7f-4374-a108-868c110c64fa admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:37.376811 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5a2b4-3c7f-4374-a108-868c110c64fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:37.377088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5a2b4-3c7f-4374-a108-868c110c64fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:37.377730 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5a2b4-3c7f-4374-a108-868c110c64fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:37.379323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5a2b4-3c7f-4374-a108-868c110c64fa admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:37.390368 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1329/2660] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:37.394467 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b589e87-2fcc-4bb1-8b16-e1ee5fcbb8e4 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:37.404446 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b589e87-2fcc-4bb1-8b16-e1ee5fcbb8e4 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:37.409089 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41186b8-e441-49d0-a12e-0c2aa193dc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:37.409733 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41186b8-e441-49d0-a12e-0c2aa193dc72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:37.410013 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41186b8-e441-49d0-a12e-0c2aa193dc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:37.410284 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41186b8-e441-49d0-a12e-0c2aa193dc72 admin admin] PATH_INFO: `/v3/projects/941839232eed44ea9fca553702bd1aeb/users/786a38ace3e248ba845349c39ee41c29/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:37.425515 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41186b8-e441-49d0-a12e-0c2aa193dc72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=941839232eed44ea9fca553702bd1aeb, user_id=786a38ace3e248ba845349c39ee41c29, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:37.428252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41186b8-e441-49d0-a12e-0c2aa193dc72 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:37.441526 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1332/2661] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:37 2022] PUT /identity/v3/projects/941839232eed44ea9fca553702bd1aeb/users/786a38ace3e248ba845349c39ee41c29/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:37.445402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5a2b4-3c7f-4374-a108-868c110c64fa admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:37.456653 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5a2b4-3c7f-4374-a108-868c110c64fa admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:37.517966 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:37.528853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa57956-357f-447d-a96c-146cbcb223f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:37.531118 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa57956-357f-447d-a96c-146cbcb223f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:37.531717 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa57956-357f-447d-a96c-146cbcb223f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:37.532073 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa57956-357f-447d-a96c-146cbcb223f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:37.532952 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa57956-357f-447d-a96c-146cbcb223f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:37.534773 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa57956-357f-447d-a96c-146cbcb223f4 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:37.548806 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1330/2662] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:37 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:37.549965 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41186b8-e441-49d0-a12e-0c2aa193dc72 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:37.562704 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41186b8-e441-49d0-a12e-0c2aa193dc72 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:37.567735 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859e2b2-6ece-4d01-ac3a-8b3732f2509f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:37.568493 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859e2b2-6ece-4d01-ac3a-8b3732f2509f admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:37.568745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859e2b2-6ece-4d01-ac3a-8b3732f2509f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:37.569027 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859e2b2-6ece-4d01-ac3a-8b3732f2509f admin admin] PATH_INFO: `/v3/projects/941839232eed44ea9fca553702bd1aeb/users/786a38ace3e248ba845349c39ee41c29/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:37.571011 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859e2b2-6ece-4d01-ac3a-8b3732f2509f admin admin] RBAC: Authorizing `identity:create_grant(project_id=941839232eed44ea9fca553702bd1aeb, user_id=786a38ace3e248ba845349c39ee41c29, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:37.575807 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859e2b2-6ece-4d01-ac3a-8b3732f2509f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:37.589132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1333/2663] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:37 2022] PUT /identity/v3/projects/941839232eed44ea9fca553702bd1aeb/users/786a38ace3e248ba845349c39ee41c29/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:16:37.600373 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccec41a-3963-4001-9702-17e1f6955ad3 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:37.600779 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccec41a-3963-4001-9702-17e1f6955ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:37.601132 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccec41a-3963-4001-9702-17e1f6955ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:37.632177 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-fccec41a-3963-4001-9702-17e1f6955ad3 None None] MFA Rules not processed for user `786a38ace3e248ba845349c39ee41c29`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:37.633650 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-fccec41a-3963-4001-9702-17e1f6955ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:37.704366 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1331/2664] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:37.715434 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859e2b2-6ece-4d01-ac3a-8b3732f2509f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:37.717209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8859e2b2-6ece-4d01-ac3a-8b3732f2509f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:37.780301 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859e2b2-6ece-4d01-ac3a-8b3732f2509f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:37.784461 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69ae74-7146-4121-af84-e5004a9d4b8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:37.785068 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69ae74-7146-4121-af84-e5004a9d4b8c service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:37.785353 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69ae74-7146-4121-af84-e5004a9d4b8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:37.785629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69ae74-7146-4121-af84-e5004a9d4b8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:37.794700 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69ae74-7146-4121-af84-e5004a9d4b8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:37.796489 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69ae74-7146-4121-af84-e5004a9d4b8c service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:37.805980 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1334/2665] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:37.838615 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccec41a-3963-4001-9702-17e1f6955ad3 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:37.907491 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccec41a-3963-4001-9702-17e1f6955ad3 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:37.912263 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db54c95-9388-4664-9f53-82b1c20161d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:37.912918 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db54c95-9388-4664-9f53-82b1c20161d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:37.913244 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db54c95-9388-4664-9f53-82b1c20161d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:37.913530 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db54c95-9388-4664-9f53-82b1c20161d6 admin admin] PATH_INFO: `/v3/users/7886cb05d5b14a628aa5e053035af818` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:37.914718 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db54c95-9388-4664-9f53-82b1c20161d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7886cb05d5b14a628aa5e053035af818)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:37.916720 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db54c95-9388-4664-9f53-82b1c20161d6 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:37.964860 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4db54c95-9388-4664-9f53-82b1c20161d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7886cb05d5b14a628aa5e053035af818'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:37.965516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4db54c95-9388-4664-9f53-82b1c20161d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7886cb05d5b14a628aa5e053035af818'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:37.973058 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4db54c95-9388-4664-9f53-82b1c20161d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7886cb05d5b14a628aa5e053035af818'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:37.980074 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4db54c95-9388-4664-9f53-82b1c20161d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7886cb05d5b14a628aa5e053035af818'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:37.980696 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-4db54c95-9388-4664-9f53-82b1c20161d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7886cb05d5b14a628aa5e053035af818'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:37.992095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:38.001690 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1332/2666] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:37 2022] DELETE /identity/v3/users/7886cb05d5b14a628aa5e053035af818 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:16:38.006223 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070a728-fbce-4446-82b5-4c474b909981 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:38.006527 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070a728-fbce-4446-82b5-4c474b909981 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:38.006788 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070a728-fbce-4446-82b5-4c474b909981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:38.020233 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:38.026319 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-5070a728-fbce-4446-82b5-4c474b909981 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:38.027516 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5070a728-fbce-4446-82b5-4c474b909981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:38.087678 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1335/2667] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:38.097378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db54c95-9388-4664-9f53-82b1c20161d6 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:38.098819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4db54c95-9388-4664-9f53-82b1c20161d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:38.157088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db54c95-9388-4664-9f53-82b1c20161d6 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:38.160860 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3fd291-7f25-44c7-8b71-d45a1c2a4919 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:38.161461 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fd291-7f25-44c7-8b71-d45a1c2a4919 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:38.161742 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fd291-7f25-44c7-8b71-d45a1c2a4919 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:38.162016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fd291-7f25-44c7-8b71-d45a1c2a4919 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:38.171455 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3fd291-7f25-44c7-8b71-d45a1c2a4919 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:38.173001 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3fd291-7f25-44c7-8b71-d45a1c2a4919 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:38.182333 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1333/2668] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:38.451821 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070a728-fbce-4446-82b5-4c474b909981 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:38.453386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5070a728-fbce-4446-82b5-4c474b909981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:38.458833 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3fd291-7f25-44c7-8b71-d45a1c2a4919 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:38.460478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7d3fd291-7f25-44c7-8b71-d45a1c2a4919 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:38.475948 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070a728-fbce-4446-82b5-4c474b909981 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:38.476453 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3fd291-7f25-44c7-8b71-d45a1c2a4919 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:38.487542 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182041d-8f50-47fd-b742-cd6b0cedc789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:38.488238 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182041d-8f50-47fd-b742-cd6b0cedc789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:38.488510 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182041d-8f50-47fd-b742-cd6b0cedc789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:38.488954 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182041d-8f50-47fd-b742-cd6b0cedc789 admin admin] PATH_INFO: `/v3/users/fba1e335998f4e518dea0102e7ff3243` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:38.490236 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182041d-8f50-47fd-b742-cd6b0cedc789 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fba1e335998f4e518dea0102e7ff3243)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:38.492652 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182041d-8f50-47fd-b742-cd6b0cedc789 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:38.493393 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff0c2e-1252-42bb-b5e3-3a7d7461e769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:38.494051 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff0c2e-1252-42bb-b5e3-3a7d7461e769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:38.494479 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff0c2e-1252-42bb-b5e3-3a7d7461e769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:38.494623 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff0c2e-1252-42bb-b5e3-3a7d7461e769 admin admin] PATH_INFO: `/v3/projects/0f7287858f5048ad9d1d9565037cfb62` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:38.495888 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff0c2e-1252-42bb-b5e3-3a7d7461e769 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f7287858f5048ad9d1d9565037cfb62)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:38.497801 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff0c2e-1252-42bb-b5e3-3a7d7461e769 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:38.540376 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f182041d-8f50-47fd-b742-cd6b0cedc789 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fba1e335998f4e518dea0102e7ff3243'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:38.541023 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f182041d-8f50-47fd-b742-cd6b0cedc789 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fba1e335998f4e518dea0102e7ff3243'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:38.545553 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d8ff0c2e-1252-42bb-b5e3-3a7d7461e769 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f7287858f5048ad9d1d9565037cfb62', 'request_id': 'req-d8ff0c2e-1252-42bb-b5e3-3a7d7461e769'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:38.549446 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f182041d-8f50-47fd-b742-cd6b0cedc789 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fba1e335998f4e518dea0102e7ff3243'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:38.550261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f182041d-8f50-47fd-b742-cd6b0cedc789 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fba1e335998f4e518dea0102e7ff3243'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:38.553731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d8ff0c2e-1252-42bb-b5e3-3a7d7461e769 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f7287858f5048ad9d1d9565037cfb62', 'request_id': 'req-d8ff0c2e-1252-42bb-b5e3-3a7d7461e769'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:38.557730 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f182041d-8f50-47fd-b742-cd6b0cedc789 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fba1e335998f4e518dea0102e7ff3243'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:38.560275 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d8ff0c2e-1252-42bb-b5e3-3a7d7461e769 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f7287858f5048ad9d1d9565037cfb62', 'request_id': 'req-d8ff0c2e-1252-42bb-b5e3-3a7d7461e769'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:38.573332 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d8ff0c2e-1252-42bb-b5e3-3a7d7461e769 admin admin] The token cache is being invalidate because project 0f7287858f5048ad9d1d9565037cfb62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:38.573750 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d8ff0c2e-1252-42bb-b5e3-3a7d7461e769 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:38.574396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d8ff0c2e-1252-42bb-b5e3-3a7d7461e769 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:38.574974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1336/2669] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:38 2022] DELETE /identity/v3/users/fba1e335998f4e518dea0102e7ff3243 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:38.578348 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1334/2670] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:38 2022] DELETE /identity/v3/projects/0f7287858f5048ad9d1d9565037cfb62 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:16:38.579988 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bcfb9-927c-444c-9eb3-b4bcca431119 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:38.580450 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff0c2e-1252-42bb-b5e3-3a7d7461e769 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:38.580681 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bcfb9-927c-444c-9eb3-b4bcca431119 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:38.580681 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bcfb9-927c-444c-9eb3-b4bcca431119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:38.581842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d8ff0c2e-1252-42bb-b5e3-3a7d7461e769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:38.605552 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-6f5bcfb9-927c-444c-9eb3-b4bcca431119 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:38.607119 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6f5bcfb9-927c-444c-9eb3-b4bcca431119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:38.651390 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff0c2e-1252-42bb-b5e3-3a7d7461e769 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:38.680549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1337/2671] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:16:38.689451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5bcfb9-927c-444c-9eb3-b4bcca431119 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:38.691677 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-6f5bcfb9-927c-444c-9eb3-b4bcca431119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:38.692822 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:38.721086 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:38.737867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bcf5fb-0c15-4e92-bbf2-a124c78c668c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:38.737867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bcf5fb-0c15-4e92-bbf2-a124c78c668c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:38.737867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bcf5fb-0c15-4e92-bbf2-a124c78c668c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:38.737867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bcf5fb-0c15-4e92-bbf2-a124c78c668c admin admin] PATH_INFO: `/v3/users/786a38ace3e248ba845349c39ee41c29` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:38.738292 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bcf5fb-0c15-4e92-bbf2-a124c78c668c admin admin] RBAC: Authorizing `identity:delete_user(user_id=786a38ace3e248ba845349c39ee41c29)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:38.738419 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bcf5fb-0c15-4e92-bbf2-a124c78c668c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:38.764989 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5bcfb9-927c-444c-9eb3-b4bcca431119 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:38.769211 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a43d9-ab67-4df0-a8e0-3fd2e159ef74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:38.769862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a43d9-ab67-4df0-a8e0-3fd2e159ef74 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:38.770233 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a43d9-ab67-4df0-a8e0-3fd2e159ef74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:38.770609 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a43d9-ab67-4df0-a8e0-3fd2e159ef74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:38.781111 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a43d9-ab67-4df0-a8e0-3fd2e159ef74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:38.782554 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a43d9-ab67-4df0-a8e0-3fd2e159ef74 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:38.786860 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-35bcf5fb-0c15-4e92-bbf2-a124c78c668c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '786a38ace3e248ba845349c39ee41c29'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:38.787614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-35bcf5fb-0c15-4e92-bbf2-a124c78c668c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '786a38ace3e248ba845349c39ee41c29'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:38.792555 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1338/2672] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:16:38.796027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-35bcf5fb-0c15-4e92-bbf2-a124c78c668c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '786a38ace3e248ba845349c39ee41c29'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:38.803298 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-35bcf5fb-0c15-4e92-bbf2-a124c78c668c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '786a38ace3e248ba845349c39ee41c29'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:38.804256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-35bcf5fb-0c15-4e92-bbf2-a124c78c668c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '786a38ace3e248ba845349c39ee41c29'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:38.824991 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1335/2673] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:38 2022] DELETE /identity/v3/users/786a38ace3e248ba845349c39ee41c29 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:39.084560 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a43d9-ab67-4df0-a8e0-3fd2e159ef74 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:39.086716 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d04a43d9-ab67-4df0-a8e0-3fd2e159ef74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:39.113405 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bcf5fb-0c15-4e92-bbf2-a124c78c668c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:39.115117 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-35bcf5fb-0c15-4e92-bbf2-a124c78c668c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:39.119032 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a43d9-ab67-4df0-a8e0-3fd2e159ef74 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:39.187062 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bcf5fb-0c15-4e92-bbf2-a124c78c668c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:39.191011 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb70165-a660-4f64-bdd4-0871c215c685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:39.191675 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb70165-a660-4f64-bdd4-0871c215c685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:39.191953 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb70165-a660-4f64-bdd4-0871c215c685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:39.192221 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb70165-a660-4f64-bdd4-0871c215c685 admin admin] PATH_INFO: `/v3/projects/c4a21c7763de40b2a48c87cd5b70bd57` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:39.192461 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a6607-30a0-4091-a836-d195938648a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:39.193087 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a6607-30a0-4091-a836-d195938648a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:39.193351 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb70165-a660-4f64-bdd4-0871c215c685 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4a21c7763de40b2a48c87cd5b70bd57)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:39.193454 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a6607-30a0-4091-a836-d195938648a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:39.193625 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a6607-30a0-4091-a836-d195938648a8 admin admin] PATH_INFO: `/v3/projects/941839232eed44ea9fca553702bd1aeb` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:39.194732 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a6607-30a0-4091-a836-d195938648a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=941839232eed44ea9fca553702bd1aeb)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:39.195275 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb70165-a660-4f64-bdd4-0871c215c685 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:39.196692 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a6607-30a0-4091-a836-d195938648a8 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:39.244577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-527a6607-30a0-4091-a836-d195938648a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '941839232eed44ea9fca553702bd1aeb', 'request_id': 'req-527a6607-30a0-4091-a836-d195938648a8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:39.244809 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7fb70165-a660-4f64-bdd4-0871c215c685 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4a21c7763de40b2a48c87cd5b70bd57', 'request_id': 'req-7fb70165-a660-4f64-bdd4-0871c215c685'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:39.246382 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:39.253280 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-527a6607-30a0-4091-a836-d195938648a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '941839232eed44ea9fca553702bd1aeb', 'request_id': 'req-527a6607-30a0-4091-a836-d195938648a8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:39.256068 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7fb70165-a660-4f64-bdd4-0871c215c685 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4a21c7763de40b2a48c87cd5b70bd57', 'request_id': 'req-7fb70165-a660-4f64-bdd4-0871c215c685'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:39.261161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-527a6607-30a0-4091-a836-d195938648a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '941839232eed44ea9fca553702bd1aeb', 'request_id': 'req-527a6607-30a0-4091-a836-d195938648a8'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:39.262193 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7fb70165-a660-4f64-bdd4-0871c215c685 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4a21c7763de40b2a48c87cd5b70bd57', 'request_id': 'req-7fb70165-a660-4f64-bdd4-0871c215c685'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:39.273415 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-527a6607-30a0-4091-a836-d195938648a8 admin admin] The token cache is being invalidate because project 941839232eed44ea9fca553702bd1aeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:39.273879 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-527a6607-30a0-4091-a836-d195938648a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:39.274525 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-527a6607-30a0-4091-a836-d195938648a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:39.274804 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7fb70165-a660-4f64-bdd4-0871c215c685 admin admin] The token cache is being invalidate because project c4a21c7763de40b2a48c87cd5b70bd57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:39.275102 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7fb70165-a660-4f64-bdd4-0871c215c685 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:39.275764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7fb70165-a660-4f64-bdd4-0871c215c685 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:39.276176 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1336/2674] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:39 2022] DELETE /identity/v3/projects/941839232eed44ea9fca553702bd1aeb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:16:39.277324 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1339/2675] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:39 2022] DELETE /identity/v3/projects/c4a21c7763de40b2a48c87cd5b70bd57 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:39.281677 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c1a7e-5c11-4d10-ae2b-86b1838ad101 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:39.281819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb70165-a660-4f64-bdd4-0871c215c685 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:39.281966 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c1a7e-5c11-4d10-ae2b-86b1838ad101 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:39.282247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c1a7e-5c11-4d10-ae2b-86b1838ad101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:39.283110 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7fb70165-a660-4f64-bdd4-0871c215c685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:39.302491 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e50c1a7e-5c11-4d10-ae2b-86b1838ad101 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:39.303708 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e50c1a7e-5c11-4d10-ae2b-86b1838ad101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:39.338079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:39.352876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb70165-a660-4f64-bdd4-0871c215c685 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:39.370920 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1337/2676] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:16:39.379510 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89f862-9b9b-462c-9c9f-04d4a7bc46bb None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:39.379824 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89f862-9b9b-462c-9c9f-04d4a7bc46bb None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:39.380108 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89f862-9b9b-462c-9c9f-04d4a7bc46bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:39.401773 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-0b89f862-9b9b-462c-9c9f-04d4a7bc46bb None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:39.403328 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0b89f862-9b9b-462c-9c9f-04d4a7bc46bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:39.413877 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1338/2677] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:16:39.418010 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89f862-9b9b-462c-9c9f-04d4a7bc46bb None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:39.418764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e7739b-7c1b-4982-8f7e-402ad902d775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:39.419416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7739b-7c1b-4982-8f7e-402ad902d775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:39.419731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7739b-7c1b-4982-8f7e-402ad902d775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:39.420015 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7739b-7c1b-4982-8f7e-402ad902d775 admin admin] PATH_INFO: `/v3/users/cc7fcc054fea4baa9301432abe697aa7` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:39.421718 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e7739b-7c1b-4982-8f7e-402ad902d775 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc7fcc054fea4baa9301432abe697aa7)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:39.423730 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e7739b-7c1b-4982-8f7e-402ad902d775 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:39.428312 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89f862-9b9b-462c-9c9f-04d4a7bc46bb None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:39.432996 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469351c-38ff-4cc7-a1ef-3864aa540aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:39.433696 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469351c-38ff-4cc7-a1ef-3864aa540aef admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:39.433979 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469351c-38ff-4cc7-a1ef-3864aa540aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:39.434252 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469351c-38ff-4cc7-a1ef-3864aa540aef admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:39.434898 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2469351c-38ff-4cc7-a1ef-3864aa540aef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:16:39.435206 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2469351c-38ff-4cc7-a1ef-3864aa540aef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:39.437223 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2469351c-38ff-4cc7-a1ef-3864aa540aef admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:39.449083 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1339/2678] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:16:39.454327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469351c-38ff-4cc7-a1ef-3864aa540aef admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:39.465294 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469351c-38ff-4cc7-a1ef-3864aa540aef admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:39.470723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d70a61-397a-4108-befd-3dd7f1fd35d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:39.471321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d70a61-397a-4108-befd-3dd7f1fd35d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:39.471628 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d70a61-397a-4108-befd-3dd7f1fd35d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:39.471905 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d70a61-397a-4108-befd-3dd7f1fd35d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:39.472548 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d70a61-397a-4108-befd-3dd7f1fd35d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:39.474456 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d70a61-397a-4108-befd-3dd7f1fd35d9 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:39.482990 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b6e7739b-7c1b-4982-8f7e-402ad902d775 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc7fcc054fea4baa9301432abe697aa7'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:39.483984 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b6e7739b-7c1b-4982-8f7e-402ad902d775 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc7fcc054fea4baa9301432abe697aa7'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:39.493318 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b6e7739b-7c1b-4982-8f7e-402ad902d775 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc7fcc054fea4baa9301432abe697aa7'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:39.494105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b6e7739b-7c1b-4982-8f7e-402ad902d775 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc7fcc054fea4baa9301432abe697aa7'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:39.502483 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b6e7739b-7c1b-4982-8f7e-402ad902d775 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc7fcc054fea4baa9301432abe697aa7'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:39.503141 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1340/2679] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:39 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:39.507227 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d70a61-397a-4108-befd-3dd7f1fd35d9 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:39.509168 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b3d70a61-397a-4108-befd-3dd7f1fd35d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:39.523667 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1340/2680] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:39 2022] DELETE /identity/v3/users/cc7fcc054fea4baa9301432abe697aa7 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:16:39.573466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d70a61-397a-4108-befd-3dd7f1fd35d9 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:39.616261 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:39.635802 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dfe527-6d62-4c3b-956a-facc4931a2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:39.636454 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dfe527-6d62-4c3b-956a-facc4931a2f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:39.636777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dfe527-6d62-4c3b-956a-facc4931a2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:39.637057 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dfe527-6d62-4c3b-956a-facc4931a2f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:39.637847 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dfe527-6d62-4c3b-956a-facc4931a2f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:39.639872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dfe527-6d62-4c3b-956a-facc4931a2f9 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:39.665022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1341/2681] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:39 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:16:39.666315 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e7739b-7c1b-4982-8f7e-402ad902d775 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:39.682067 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e7739b-7c1b-4982-8f7e-402ad902d775 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:39.687834 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f1777a-c182-4547-a4cc-107bd8a8cfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:39.688496 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1777a-c182-4547-a4cc-107bd8a8cfd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:39.688779 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1777a-c182-4547-a4cc-107bd8a8cfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:39.689045 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1777a-c182-4547-a4cc-107bd8a8cfd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:39.689688 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f1777a-c182-4547-a4cc-107bd8a8cfd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:39.691487 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f1777a-c182-4547-a4cc-107bd8a8cfd3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:39.701599 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1341/2682] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:39.705811 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dfe527-6d62-4c3b-956a-facc4931a2f9 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:39.715714 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dfe527-6d62-4c3b-956a-facc4931a2f9 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:39.720825 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d32956-cbfa-4d09-8a74-5299a2406302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:39.721436 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d32956-cbfa-4d09-8a74-5299a2406302 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:39.721713 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d32956-cbfa-4d09-8a74-5299a2406302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:39.722305 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d32956-cbfa-4d09-8a74-5299a2406302 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:39.722617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d32956-cbfa-4d09-8a74-5299a2406302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:39.724251 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d32956-cbfa-4d09-8a74-5299a2406302 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:39.735130 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1342/2683] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:39.739351 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f1777a-c182-4547-a4cc-107bd8a8cfd3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:39.751525 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f1777a-c182-4547-a4cc-107bd8a8cfd3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:39.756905 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578118d-dfb2-434b-b690-7708e9a85f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:39.757753 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578118d-dfb2-434b-b690-7708e9a85f30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:39.757932 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578118d-dfb2-434b-b690-7708e9a85f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:39.758778 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578118d-dfb2-434b-b690-7708e9a85f30 admin admin] PATH_INFO: `/v3/projects/798a26d1b86541e2b8aa5f4d7b8a1cc2/users/f7cf7384c60d49988dd98d47f716be94/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:39.764594 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d32956-cbfa-4d09-8a74-5299a2406302 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:39.766315 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e3d32956-cbfa-4d09-8a74-5299a2406302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:39.776588 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9578118d-dfb2-434b-b690-7708e9a85f30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=798a26d1b86541e2b8aa5f4d7b8a1cc2, user_id=f7cf7384c60d49988dd98d47f716be94, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:39.779472 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d32956-cbfa-4d09-8a74-5299a2406302 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:39.779657 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9578118d-dfb2-434b-b690-7708e9a85f30 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:39.785462 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ed745-546f-4673-b304-c99e4d881e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:39.786199 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ed745-546f-4673-b304-c99e4d881e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:39.786482 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ed745-546f-4673-b304-c99e4d881e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:39.786765 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ed745-546f-4673-b304-c99e4d881e85 admin admin] PATH_INFO: `/v3/users/a810826b0d62473481cccf432a7d5dbb` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:39.787967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ed745-546f-4673-b304-c99e4d881e85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a810826b0d62473481cccf432a7d5dbb)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:39.790133 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ed745-546f-4673-b304-c99e4d881e85 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:39.801912 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1342/2684] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:39 2022] PUT /identity/v3/projects/798a26d1b86541e2b8aa5f4d7b8a1cc2/users/f7cf7384c60d49988dd98d47f716be94/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:39.811571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578118d-dfb2-434b-b690-7708e9a85f30 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:39.814209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9578118d-dfb2-434b-b690-7708e9a85f30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:39.860727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-626ed745-546f-4673-b304-c99e4d881e85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a810826b0d62473481cccf432a7d5dbb'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:39.861497 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-626ed745-546f-4673-b304-c99e4d881e85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a810826b0d62473481cccf432a7d5dbb'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:39.871901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-626ed745-546f-4673-b304-c99e4d881e85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a810826b0d62473481cccf432a7d5dbb'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:39.877996 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:39.879388 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-626ed745-546f-4673-b304-c99e4d881e85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a810826b0d62473481cccf432a7d5dbb'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:39.880328 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-626ed745-546f-4673-b304-c99e4d881e85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a810826b0d62473481cccf432a7d5dbb'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:39.898523 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578118d-dfb2-434b-b690-7708e9a85f30 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:39.900423 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1343/2685] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:39 2022] DELETE /identity/v3/users/a810826b0d62473481cccf432a7d5dbb => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:16:39.901352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366d62d-b217-4c29-a15c-900dc60d7774 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:39.901638 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366d62d-b217-4c29-a15c-900dc60d7774 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:39.901913 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366d62d-b217-4c29-a15c-900dc60d7774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:39.931840 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d366d62d-b217-4c29-a15c-900dc60d7774 None None] MFA Rules not processed for user `f7cf7384c60d49988dd98d47f716be94`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:39.933050 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d366d62d-b217-4c29-a15c-900dc60d7774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:39.958683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4696f8-b0ce-4671-8595-443a91921c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:39.959240 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4696f8-b0ce-4671-8595-443a91921c14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:39.959523 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4696f8-b0ce-4671-8595-443a91921c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:39.959832 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4696f8-b0ce-4671-8595-443a91921c14 admin admin] PATH_INFO: `/v3/projects/d4212d11ac6b4afda1ac59e360d9e51e` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:39.961012 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4696f8-b0ce-4671-8595-443a91921c14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4212d11ac6b4afda1ac59e360d9e51e)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:39.962932 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4696f8-b0ce-4671-8595-443a91921c14 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:40.004735 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1344/2686] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:40.005977 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fec158-5ff8-4ce3-86c8-ad927f04a249 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:40.006275 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fec158-5ff8-4ce3-86c8-ad927f04a249 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:40.006562 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fec158-5ff8-4ce3-86c8-ad927f04a249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:40.022958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-da4696f8-b0ce-4671-8595-443a91921c14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4212d11ac6b4afda1ac59e360d9e51e', 'request_id': 'req-da4696f8-b0ce-4671-8595-443a91921c14'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:40.026195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-54fec158-5ff8-4ce3-86c8-ad927f04a249 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:40.027421 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-54fec158-5ff8-4ce3-86c8-ad927f04a249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:40.032490 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-da4696f8-b0ce-4671-8595-443a91921c14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4212d11ac6b4afda1ac59e360d9e51e', 'request_id': 'req-da4696f8-b0ce-4671-8595-443a91921c14'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:40.039685 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-da4696f8-b0ce-4671-8595-443a91921c14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4212d11ac6b4afda1ac59e360d9e51e', 'request_id': 'req-da4696f8-b0ce-4671-8595-443a91921c14'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:40.049150 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:40.056217 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-da4696f8-b0ce-4671-8595-443a91921c14 admin admin] The token cache is being invalidate because project d4212d11ac6b4afda1ac59e360d9e51e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:40.056511 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-da4696f8-b0ce-4671-8595-443a91921c14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:40.057274 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-da4696f8-b0ce-4671-8595-443a91921c14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:40.058874 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1343/2687] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:39 2022] DELETE /identity/v3/projects/d4212d11ac6b4afda1ac59e360d9e51e => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:16:40.059618 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4696f8-b0ce-4671-8595-443a91921c14 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:40.060938 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-da4696f8-b0ce-4671-8595-443a91921c14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:40.100904 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1345/2688] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:40.101584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fec158-5ff8-4ce3-86c8-ad927f04a249 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:40.102993 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-54fec158-5ff8-4ce3-86c8-ad927f04a249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:40.122293 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fec158-5ff8-4ce3-86c8-ad927f04a249 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:40.129563 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0748e-991d-457c-ad30-4906c61c8b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:40.131464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0748e-991d-457c-ad30-4906c61c8b12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:40.131464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0748e-991d-457c-ad30-4906c61c8b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:40.131464 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0748e-991d-457c-ad30-4906c61c8b12 admin admin] PATH_INFO: `/v3/users/56a7b2ddd7b24a83adc885754dea95c1` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:40.133197 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e0748e-991d-457c-ad30-4906c61c8b12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56a7b2ddd7b24a83adc885754dea95c1)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:40.136863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e0748e-991d-457c-ad30-4906c61c8b12 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:40.164795 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4696f8-b0ce-4671-8595-443a91921c14 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:40.170229 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587ab3d-3934-4ead-af90-2d775380de1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:40.171848 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587ab3d-3934-4ead-af90-2d775380de1f admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:40.172286 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587ab3d-3934-4ead-af90-2d775380de1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:40.172707 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587ab3d-3934-4ead-af90-2d775380de1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:40.173495 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587ab3d-3934-4ead-af90-2d775380de1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:40.176229 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587ab3d-3934-4ead-af90-2d775380de1f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:40.220212 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1344/2689] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:40 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:40.221186 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-34e0748e-991d-457c-ad30-4906c61c8b12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56a7b2ddd7b24a83adc885754dea95c1'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:40.221588 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587ab3d-3934-4ead-af90-2d775380de1f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:40.222284 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-34e0748e-991d-457c-ad30-4906c61c8b12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56a7b2ddd7b24a83adc885754dea95c1'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:40.224535 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1587ab3d-3934-4ead-af90-2d775380de1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:40.236685 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-34e0748e-991d-457c-ad30-4906c61c8b12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56a7b2ddd7b24a83adc885754dea95c1'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:40.239350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:40.247452 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-34e0748e-991d-457c-ad30-4906c61c8b12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56a7b2ddd7b24a83adc885754dea95c1'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:40.248509 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-34e0748e-991d-457c-ad30-4906c61c8b12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56a7b2ddd7b24a83adc885754dea95c1'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:40.269648 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1346/2690] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:40 2022] DELETE /identity/v3/users/56a7b2ddd7b24a83adc885754dea95c1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:40.272480 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0748e-991d-457c-ad30-4906c61c8b12 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:40.273990 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-34e0748e-991d-457c-ad30-4906c61c8b12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:40.308830 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587ab3d-3934-4ead-af90-2d775380de1f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:40.384946 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0748e-991d-457c-ad30-4906c61c8b12 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:40.385454 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a27c7-e01c-478a-a176-6850dd69a06b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:40.386077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a27c7-e01c-478a-a176-6850dd69a06b service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:40.386361 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a27c7-e01c-478a-a176-6850dd69a06b service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:40.386638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a27c7-e01c-478a-a176-6850dd69a06b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:40.388648 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-028a27c7-e01c-478a-a176-6850dd69a06b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:40.390987 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951db25b-a0e6-4bde-acca-e21fe535dca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:40.391822 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951db25b-a0e6-4bde-acca-e21fe535dca5 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:40.392101 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951db25b-a0e6-4bde-acca-e21fe535dca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:40.392372 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951db25b-a0e6-4bde-acca-e21fe535dca5 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:40.393112 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951db25b-a0e6-4bde-acca-e21fe535dca5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:40.395055 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951db25b-a0e6-4bde-acca-e21fe535dca5 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:40.406836 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a27c7-e01c-478a-a176-6850dd69a06b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:40.410550 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a27c7-e01c-478a-a176-6850dd69a06b service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:40.424140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1347/2691] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:40 2022] POST /identity/v3/users => generated 300 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:40.425061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1345/2692] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:40.428239 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951db25b-a0e6-4bde-acca-e21fe535dca5 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:40.439864 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951db25b-a0e6-4bde-acca-e21fe535dca5 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:40.445363 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb83ba87-19b8-46d8-9762-b0bd5641f542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:40.446242 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83ba87-19b8-46d8-9762-b0bd5641f542 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:40.446434 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83ba87-19b8-46d8-9762-b0bd5641f542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:40.447075 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83ba87-19b8-46d8-9762-b0bd5641f542 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:40.447379 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb83ba87-19b8-46d8-9762-b0bd5641f542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:40.449463 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb83ba87-19b8-46d8-9762-b0bd5641f542 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:40.462860 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1348/2693] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:16:40.467407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a27c7-e01c-478a-a176-6850dd69a06b service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:40.479296 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a27c7-e01c-478a-a176-6850dd69a06b service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:40.484867 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747437d-38d3-4e65-a53b-cb79ae023422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:40.485492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747437d-38d3-4e65-a53b-cb79ae023422 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:40.485784 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747437d-38d3-4e65-a53b-cb79ae023422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:40.486064 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747437d-38d3-4e65-a53b-cb79ae023422 admin admin] PATH_INFO: `/v3/projects/42d42f56546d4ef69a258ffb5edccac1/users/a575680063614186b324a7f93fef8d5c/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:40.503311 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4747437d-38d3-4e65-a53b-cb79ae023422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42d42f56546d4ef69a258ffb5edccac1, user_id=a575680063614186b324a7f93fef8d5c, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:40.506705 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4747437d-38d3-4e65-a53b-cb79ae023422 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:40.533765 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1346/2694] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:40 2022] PUT /identity/v3/projects/42d42f56546d4ef69a258ffb5edccac1/users/a575680063614186b324a7f93fef8d5c/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:40.538565 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb83ba87-19b8-46d8-9762-b0bd5641f542 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:40.551209 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb83ba87-19b8-46d8-9762-b0bd5641f542 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:40.566036 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:40.628135 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b381a7-7afd-4f4b-a0a2-0c85439fd7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:40.628858 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b381a7-7afd-4f4b-a0a2-0c85439fd7b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:40.629139 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b381a7-7afd-4f4b-a0a2-0c85439fd7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:40.629411 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b381a7-7afd-4f4b-a0a2-0c85439fd7b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:40.630057 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b381a7-7afd-4f4b-a0a2-0c85439fd7b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:40.631725 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b381a7-7afd-4f4b-a0a2-0c85439fd7b4 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:40.643777 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1349/2695] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:40.649090 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747437d-38d3-4e65-a53b-cb79ae023422 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:40.661080 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747437d-38d3-4e65-a53b-cb79ae023422 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:40.667697 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741ee80-a6a6-41e2-9622-1a630b88a2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:40.668770 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741ee80-a6a6-41e2-9622-1a630b88a2a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:40.669081 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741ee80-a6a6-41e2-9622-1a630b88a2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:40.669351 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741ee80-a6a6-41e2-9622-1a630b88a2a7 admin admin] PATH_INFO: `/v3/projects/42d42f56546d4ef69a258ffb5edccac1/users/a575680063614186b324a7f93fef8d5c/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:40.671437 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a741ee80-a6a6-41e2-9622-1a630b88a2a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42d42f56546d4ef69a258ffb5edccac1, user_id=a575680063614186b324a7f93fef8d5c, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:40.674497 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a741ee80-a6a6-41e2-9622-1a630b88a2a7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:40.686001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1347/2696] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:40 2022] PUT /identity/v3/projects/42d42f56546d4ef69a258ffb5edccac1/users/a575680063614186b324a7f93fef8d5c/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:16:40.698328 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc5a7d-7217-4d21-8726-9b1a5625ceaf None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:40.698614 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc5a7d-7217-4d21-8726-9b1a5625ceaf None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:40.698884 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc5a7d-7217-4d21-8726-9b1a5625ceaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:40.747510 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741ee80-a6a6-41e2-9622-1a630b88a2a7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:40.747822 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-10cc5a7d-7217-4d21-8726-9b1a5625ceaf None None] MFA Rules not processed for user `a575680063614186b324a7f93fef8d5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:40.749913 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a741ee80-a6a6-41e2-9622-1a630b88a2a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:40.752009 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-10cc5a7d-7217-4d21-8726-9b1a5625ceaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:40.853712 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741ee80-a6a6-41e2-9622-1a630b88a2a7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:40.857721 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1350/2697] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:40 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:40.858633 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5c0ab-1eae-4713-8a1a-80343a639cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:40.859670 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5c0ab-1eae-4713-8a1a-80343a639cc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:40.860185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5c0ab-1eae-4713-8a1a-80343a639cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:40.860351 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5c0ab-1eae-4713-8a1a-80343a639cc4 admin admin] PATH_INFO: `/v3/projects/0de7d515d7564e82944640dc066acabf` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:40.861554 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5c0ab-1eae-4713-8a1a-80343a639cc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0de7d515d7564e82944640dc066acabf)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:40.863896 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5c0ab-1eae-4713-8a1a-80343a639cc4 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:40.870160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc5a7d-7217-4d21-8726-9b1a5625ceaf None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:40.872921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-10cc5a7d-7217-4d21-8726-9b1a5625ceaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:40.911404 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:40.929590 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c4b5c0ab-1eae-4713-8a1a-80343a639cc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0de7d515d7564e82944640dc066acabf', 'request_id': 'req-c4b5c0ab-1eae-4713-8a1a-80343a639cc4'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:40.941115 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c4b5c0ab-1eae-4713-8a1a-80343a639cc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0de7d515d7564e82944640dc066acabf', 'request_id': 'req-c4b5c0ab-1eae-4713-8a1a-80343a639cc4'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:40.946277 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc5a7d-7217-4d21-8726-9b1a5625ceaf None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:40.956775 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c4b5c0ab-1eae-4713-8a1a-80343a639cc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0de7d515d7564e82944640dc066acabf', 'request_id': 'req-c4b5c0ab-1eae-4713-8a1a-80343a639cc4'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:40.972874 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c4b5c0ab-1eae-4713-8a1a-80343a639cc4 admin admin] The token cache is being invalidate because project 0de7d515d7564e82944640dc066acabf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:40.973185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c4b5c0ab-1eae-4713-8a1a-80343a639cc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:40.973922 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-c4b5c0ab-1eae-4713-8a1a-80343a639cc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:40.975433 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1348/2698] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:40 2022] DELETE /identity/v3/projects/0de7d515d7564e82944640dc066acabf => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:16:40.976322 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5c0ab-1eae-4713-8a1a-80343a639cc4 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:40.977712 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c4b5c0ab-1eae-4713-8a1a-80343a639cc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:41.000451 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0e4174-5748-4d2f-83b5-5d9efd1364de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:41.001067 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e4174-5748-4d2f-83b5-5d9efd1364de service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:41.001354 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e4174-5748-4d2f-83b5-5d9efd1364de service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:41.001629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e4174-5748-4d2f-83b5-5d9efd1364de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:41.003369 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-bd0e4174-5748-4d2f-83b5-5d9efd1364de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:41.010267 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:41.079383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0e4174-5748-4d2f-83b5-5d9efd1364de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:41.080898 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0e4174-5748-4d2f-83b5-5d9efd1364de service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:41.091909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1351/2699] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:40 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:16:41.093270 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e5a21-5aac-41b6-9c4c-0cb2c763a130 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:41.093593 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e5a21-5aac-41b6-9c4c-0cb2c763a130 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:41.093970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e5a21-5aac-41b6-9c4c-0cb2c763a130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:41.115584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-fe8e5a21-5aac-41b6-9c4c-0cb2c763a130 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:41.116841 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-fe8e5a21-5aac-41b6-9c4c-0cb2c763a130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:41.201249 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1352/2700] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:41.202319 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e5a21-5aac-41b6-9c4c-0cb2c763a130 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:41.203829 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-fe8e5a21-5aac-41b6-9c4c-0cb2c763a130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:41.212465 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5c0ab-1eae-4713-8a1a-80343a639cc4 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:41.215871 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e5a21-5aac-41b6-9c4c-0cb2c763a130 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:41.216676 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e907f-3363-4a8a-9d12-00a4b02cfdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:41.217313 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e907f-3363-4a8a-9d12-00a4b02cfdf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:41.217651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e907f-3363-4a8a-9d12-00a4b02cfdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:41.218083 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e907f-3363-4a8a-9d12-00a4b02cfdf6 admin admin] PATH_INFO: `/v3/projects/053dfa24a9884cbeb7afc6e017d02e80` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:41.219461 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086e907f-3363-4a8a-9d12-00a4b02cfdf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=053dfa24a9884cbeb7afc6e017d02e80)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:41.220724 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b86aa3-d5bb-4d88-8cf6-1f9cea213ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:41.221377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b86aa3-d5bb-4d88-8cf6-1f9cea213ec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:41.221486 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086e907f-3363-4a8a-9d12-00a4b02cfdf6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:41.221645 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b86aa3-d5bb-4d88-8cf6-1f9cea213ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:41.221928 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b86aa3-d5bb-4d88-8cf6-1f9cea213ec7 admin admin] PATH_INFO: `/v3/users/f7cf7384c60d49988dd98d47f716be94` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:41.223130 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b86aa3-d5bb-4d88-8cf6-1f9cea213ec7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7cf7384c60d49988dd98d47f716be94)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:41.225021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b86aa3-d5bb-4d88-8cf6-1f9cea213ec7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:41.276236 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-086e907f-3363-4a8a-9d12-00a4b02cfdf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '053dfa24a9884cbeb7afc6e017d02e80', 'request_id': 'req-086e907f-3363-4a8a-9d12-00a4b02cfdf6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:41.279294 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-06b86aa3-d5bb-4d88-8cf6-1f9cea213ec7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7cf7384c60d49988dd98d47f716be94'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:41.279999 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-06b86aa3-d5bb-4d88-8cf6-1f9cea213ec7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7cf7384c60d49988dd98d47f716be94'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:41.284541 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-086e907f-3363-4a8a-9d12-00a4b02cfdf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '053dfa24a9884cbeb7afc6e017d02e80', 'request_id': 'req-086e907f-3363-4a8a-9d12-00a4b02cfdf6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:41.287890 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-06b86aa3-d5bb-4d88-8cf6-1f9cea213ec7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7cf7384c60d49988dd98d47f716be94'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:41.290373 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-086e907f-3363-4a8a-9d12-00a4b02cfdf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '053dfa24a9884cbeb7afc6e017d02e80', 'request_id': 'req-086e907f-3363-4a8a-9d12-00a4b02cfdf6'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:41.294909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-06b86aa3-d5bb-4d88-8cf6-1f9cea213ec7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7cf7384c60d49988dd98d47f716be94'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:41.295569 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-06b86aa3-d5bb-4d88-8cf6-1f9cea213ec7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7cf7384c60d49988dd98d47f716be94'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:41.302884 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-086e907f-3363-4a8a-9d12-00a4b02cfdf6 admin admin] The token cache is being invalidate because project 053dfa24a9884cbeb7afc6e017d02e80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:41.303174 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-086e907f-3363-4a8a-9d12-00a4b02cfdf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:41.303845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-086e907f-3363-4a8a-9d12-00a4b02cfdf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:41.305325 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1349/2701] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:40 2022] DELETE /identity/v3/projects/053dfa24a9884cbeb7afc6e017d02e80 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:41.306497 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5655c0-71a8-4366-835a-0008782eed16 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:41.306774 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5655c0-71a8-4366-835a-0008782eed16 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:41.307043 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5655c0-71a8-4366-835a-0008782eed16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:41.318995 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1353/2702] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:41 2022] DELETE /identity/v3/users/f7cf7384c60d49988dd98d47f716be94 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:41.319883 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b86aa3-d5bb-4d88-8cf6-1f9cea213ec7 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:41.322022 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-06b86aa3-d5bb-4d88-8cf6-1f9cea213ec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:41.327573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-5d5655c0-71a8-4366-835a-0008782eed16 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:41.328952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5d5655c0-71a8-4366-835a-0008782eed16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:41.340514 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:41.374900 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:41.391809 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1350/2703] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:41.393155 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a481811-1501-4d81-8df5-a56d49b4b855 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:41.393500 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a481811-1501-4d81-8df5-a56d49b4b855 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:41.393605 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b86aa3-d5bb-4d88-8cf6-1f9cea213ec7 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:41.394002 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a481811-1501-4d81-8df5-a56d49b4b855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:41.398486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30accd33-24c4-4cf0-896d-667fede24f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:41.399113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30accd33-24c4-4cf0-896d-667fede24f63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:41.399396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30accd33-24c4-4cf0-896d-667fede24f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:41.399708 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30accd33-24c4-4cf0-896d-667fede24f63 admin admin] PATH_INFO: `/v3/users/561e1edd6aa142deadbfe4ca5ecb9d4c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:41.400851 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30accd33-24c4-4cf0-896d-667fede24f63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=561e1edd6aa142deadbfe4ca5ecb9d4c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:41.402806 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30accd33-24c4-4cf0-896d-667fede24f63 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:41.413026 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-4a481811-1501-4d81-8df5-a56d49b4b855 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:41.414210 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-4a481811-1501-4d81-8df5-a56d49b4b855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:41.422545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1351/2704] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:41.423323 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a481811-1501-4d81-8df5-a56d49b4b855 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:41.434670 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a481811-1501-4d81-8df5-a56d49b4b855 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:41.439411 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c9f46-4c12-4ea3-9804-6a89bfdc5d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:41.440100 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c9f46-4c12-4ea3-9804-6a89bfdc5d24 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:41.440392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c9f46-4c12-4ea3-9804-6a89bfdc5d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:41.440644 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c9f46-4c12-4ea3-9804-6a89bfdc5d24 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:41.441289 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7c9f46-4c12-4ea3-9804-6a89bfdc5d24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:16:41.441587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7c9f46-4c12-4ea3-9804-6a89bfdc5d24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:41.443182 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7c9f46-4c12-4ea3-9804-6a89bfdc5d24 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:41.453070 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-30accd33-24c4-4cf0-896d-667fede24f63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '561e1edd6aa142deadbfe4ca5ecb9d4c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:41.453804 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-30accd33-24c4-4cf0-896d-667fede24f63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '561e1edd6aa142deadbfe4ca5ecb9d4c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:41.455134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1352/2705] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:16:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:16:41.456028 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c9f46-4c12-4ea3-9804-6a89bfdc5d24 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:41.467102 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3a7c9f46-4c12-4ea3-9804-6a89bfdc5d24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:41.469571 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-30accd33-24c4-4cf0-896d-667fede24f63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '561e1edd6aa142deadbfe4ca5ecb9d4c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:41.481847 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-30accd33-24c4-4cf0-896d-667fede24f63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '561e1edd6aa142deadbfe4ca5ecb9d4c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:41.482996 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-30accd33-24c4-4cf0-896d-667fede24f63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '561e1edd6aa142deadbfe4ca5ecb9d4c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:41.510101 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1354/2706] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:41 2022] DELETE /identity/v3/users/561e1edd6aa142deadbfe4ca5ecb9d4c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:41.521606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30accd33-24c4-4cf0-896d-667fede24f63 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:41.523308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-30accd33-24c4-4cf0-896d-667fede24f63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:41.541569 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c9f46-4c12-4ea3-9804-6a89bfdc5d24 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:41.598947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe147f-7e11-41b5-a671-2164a259dbbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:41.599751 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe147f-7e11-41b5-a671-2164a259dbbc service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:41.600417 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe147f-7e11-41b5-a671-2164a259dbbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:41.600708 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe147f-7e11-41b5-a671-2164a259dbbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:41.604042 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-76fe147f-7e11-41b5-a671-2164a259dbbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:41.605817 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30accd33-24c4-4cf0-896d-667fede24f63 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:41.612946 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cbdb8-aafd-4ea3-b4e0-f4dad830ef41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:41.613617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cbdb8-aafd-4ea3-b4e0-f4dad830ef41 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:41.613903 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cbdb8-aafd-4ea3-b4e0-f4dad830ef41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:41.614281 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1cbdb8-aafd-4ea3-b4e0-f4dad830ef41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:41.615276 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1cbdb8-aafd-4ea3-b4e0-f4dad830ef41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:41.617478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1cbdb8-aafd-4ea3-b4e0-f4dad830ef41 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:41.618520 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe147f-7e11-41b5-a671-2164a259dbbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:41.620416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe147f-7e11-41b5-a671-2164a259dbbc service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:41.632023 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1353/2707] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:41.648377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1355/2708] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:41 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:41.652415 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe147f-7e11-41b5-a671-2164a259dbbc service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:41.663534 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe147f-7e11-41b5-a671-2164a259dbbc service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:41.732646 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76c0a0-781f-4cd6-a2b4-83107f8e1119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:41.734089 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76c0a0-781f-4cd6-a2b4-83107f8e1119 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:41.734089 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76c0a0-781f-4cd6-a2b4-83107f8e1119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:41.734559 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76c0a0-781f-4cd6-a2b4-83107f8e1119 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:41.735463 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76c0a0-781f-4cd6-a2b4-83107f8e1119 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:41.737559 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76c0a0-781f-4cd6-a2b4-83107f8e1119 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:41.761857 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1354/2709] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:41 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:41.766338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cbdb8-aafd-4ea3-b4e0-f4dad830ef41 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:41.776830 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1cbdb8-aafd-4ea3-b4e0-f4dad830ef41 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:41.782310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2da34-dc88-4db7-ab84-a0b80686ce64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:41.782931 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2da34-dc88-4db7-ab84-a0b80686ce64 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:41.783218 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2da34-dc88-4db7-ab84-a0b80686ce64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:41.783495 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2da34-dc88-4db7-ab84-a0b80686ce64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:41.784421 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2da34-dc88-4db7-ab84-a0b80686ce64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:41.786304 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2da34-dc88-4db7-ab84-a0b80686ce64 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:41.803829 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1356/2710] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:41.805040 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76c0a0-781f-4cd6-a2b4-83107f8e1119 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:41.817268 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76c0a0-781f-4cd6-a2b4-83107f8e1119 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:41.822883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1376327-24e6-4457-bd0e-b69add902acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:41.824117 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1376327-24e6-4457-bd0e-b69add902acf admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:41.824398 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1376327-24e6-4457-bd0e-b69add902acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:41.824772 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1376327-24e6-4457-bd0e-b69add902acf admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:41.825599 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1376327-24e6-4457-bd0e-b69add902acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:41.828458 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1376327-24e6-4457-bd0e-b69add902acf admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:41.841050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1355/2711] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:41.845693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2da34-dc88-4db7-ab84-a0b80686ce64 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:41.856771 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2da34-dc88-4db7-ab84-a0b80686ce64 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:41.858205 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1376327-24e6-4457-bd0e-b69add902acf admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:41.860657 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c1376327-24e6-4457-bd0e-b69add902acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:41.865810 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba80188-bf8d-46b8-a3db-9d20ed345a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:41.866485 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba80188-bf8d-46b8-a3db-9d20ed345a8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:41.866766 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba80188-bf8d-46b8-a3db-9d20ed345a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:41.867039 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba80188-bf8d-46b8-a3db-9d20ed345a8f admin admin] PATH_INFO: `/v3/projects/17c76b2507f74d758561d5d8a91322f4/users/05c420254b904aa9b7ff5b843293cbb0/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:41.870382 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:41.874669 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1376327-24e6-4457-bd0e-b69add902acf admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:41.882261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ebe0dc-8849-4240-83c1-a006ea54146d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:41.883732 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebe0dc-8849-4240-83c1-a006ea54146d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:41.884033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebe0dc-8849-4240-83c1-a006ea54146d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:41.884301 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebe0dc-8849-4240-83c1-a006ea54146d admin admin] PATH_INFO: `/v3/projects/4a04d886ed7d4f188f7b33e41e94e76f` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:41.886962 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ebe0dc-8849-4240-83c1-a006ea54146d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a04d886ed7d4f188f7b33e41e94e76f)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:41.889072 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ebe0dc-8849-4240-83c1-a006ea54146d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:41.891752 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:41.901541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba80188-bf8d-46b8-a3db-9d20ed345a8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=17c76b2507f74d758561d5d8a91322f4, user_id=05c420254b904aa9b7ff5b843293cbb0, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:41.904815 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba80188-bf8d-46b8-a3db-9d20ed345a8f admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:41.923045 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1357/2712] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:41 2022] PUT /identity/v3/projects/17c76b2507f74d758561d5d8a91322f4/users/05c420254b904aa9b7ff5b843293cbb0/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:41.933786 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beda2ae-d0ed-4e05-b1d9-ebc178692623 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:41.934072 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beda2ae-d0ed-4e05-b1d9-ebc178692623 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:41.934394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beda2ae-d0ed-4e05-b1d9-ebc178692623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:41.951823 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-06ebe0dc-8849-4240-83c1-a006ea54146d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a04d886ed7d4f188f7b33e41e94e76f', 'request_id': 'req-06ebe0dc-8849-4240-83c1-a006ea54146d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:41.965819 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-06ebe0dc-8849-4240-83c1-a006ea54146d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a04d886ed7d4f188f7b33e41e94e76f', 'request_id': 'req-06ebe0dc-8849-4240-83c1-a006ea54146d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:41.969177 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-4beda2ae-d0ed-4e05-b1d9-ebc178692623 None None] MFA Rules not processed for user `05c420254b904aa9b7ff5b843293cbb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:41.970454 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4beda2ae-d0ed-4e05-b1d9-ebc178692623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:41.971356 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-06ebe0dc-8849-4240-83c1-a006ea54146d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a04d886ed7d4f188f7b33e41e94e76f', 'request_id': 'req-06ebe0dc-8849-4240-83c1-a006ea54146d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:41.984813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-06ebe0dc-8849-4240-83c1-a006ea54146d admin admin] The token cache is being invalidate because project 4a04d886ed7d4f188f7b33e41e94e76f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:41.985223 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-06ebe0dc-8849-4240-83c1-a006ea54146d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:41.985998 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-06ebe0dc-8849-4240-83c1-a006ea54146d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:41.987848 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1356/2713] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:41 2022] DELETE /identity/v3/projects/4a04d886ed7d4f188f7b33e41e94e76f => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:16:41.992644 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ebe0dc-8849-4240-83c1-a006ea54146d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:41.994631 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-06ebe0dc-8849-4240-83c1-a006ea54146d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:42.043839 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1358/2714] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:16:42.045093 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3b167-66b8-4390-aa42-66795dba1307 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:42.045374 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3b167-66b8-4390-aa42-66795dba1307 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:42.046159 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3b167-66b8-4390-aa42-66795dba1307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:42.066101 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-24d3b167-66b8-4390-aa42-66795dba1307 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:42.067398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-24d3b167-66b8-4390-aa42-66795dba1307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:42.072650 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ebe0dc-8849-4240-83c1-a006ea54146d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:42.075539 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1359/2715] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:42.076351 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3b167-66b8-4390-aa42-66795dba1307 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:42.078604 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-24d3b167-66b8-4390-aa42-66795dba1307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:42.085152 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fffdca8-6bc2-492a-bef6-86a3a6ec9047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:42.085807 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fffdca8-6bc2-492a-bef6-86a3a6ec9047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:42.086092 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fffdca8-6bc2-492a-bef6-86a3a6ec9047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:42.086366 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fffdca8-6bc2-492a-bef6-86a3a6ec9047 admin admin] PATH_INFO: `/v3/projects/798a26d1b86541e2b8aa5f4d7b8a1cc2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:42.087543 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fffdca8-6bc2-492a-bef6-86a3a6ec9047 admin admin] RBAC: Authorizing `identity:delete_project(project_id=798a26d1b86541e2b8aa5f4d7b8a1cc2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:42.089580 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fffdca8-6bc2-492a-bef6-86a3a6ec9047 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:42.138938 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2fffdca8-6bc2-492a-bef6-86a3a6ec9047 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '798a26d1b86541e2b8aa5f4d7b8a1cc2', 'request_id': 'req-2fffdca8-6bc2-492a-bef6-86a3a6ec9047'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:42.142059 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3b167-66b8-4390-aa42-66795dba1307 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:42.147232 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2fffdca8-6bc2-492a-bef6-86a3a6ec9047 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '798a26d1b86541e2b8aa5f4d7b8a1cc2', 'request_id': 'req-2fffdca8-6bc2-492a-bef6-86a3a6ec9047'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:42.153461 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2fffdca8-6bc2-492a-bef6-86a3a6ec9047 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '798a26d1b86541e2b8aa5f4d7b8a1cc2', 'request_id': 'req-2fffdca8-6bc2-492a-bef6-86a3a6ec9047'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:42.165145 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2fffdca8-6bc2-492a-bef6-86a3a6ec9047 admin admin] The token cache is being invalidate because project 798a26d1b86541e2b8aa5f4d7b8a1cc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:42.165505 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2fffdca8-6bc2-492a-bef6-86a3a6ec9047 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:42.166169 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2fffdca8-6bc2-492a-bef6-86a3a6ec9047 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:42.167658 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1357/2716] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:41 2022] DELETE /identity/v3/projects/798a26d1b86541e2b8aa5f4d7b8a1cc2 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:42.168901 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57bbd1-5872-438a-aa1f-8647e53a24a8 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:42.169183 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57bbd1-5872-438a-aa1f-8647e53a24a8 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:42.169459 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57bbd1-5872-438a-aa1f-8647e53a24a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:42.189485 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-bd57bbd1-5872-438a-aa1f-8647e53a24a8 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:42.190678 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bd57bbd1-5872-438a-aa1f-8647e53a24a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:42.193406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945c6ca-cd7c-4a14-b411-600fbbc83c4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:42.194026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945c6ca-cd7c-4a14-b411-600fbbc83c4f service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:42.194325 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945c6ca-cd7c-4a14-b411-600fbbc83c4f service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:42.194577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945c6ca-cd7c-4a14-b411-600fbbc83c4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:42.196387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e945c6ca-cd7c-4a14-b411-600fbbc83c4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:42.204368 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:42.236442 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:42.255659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1358/2717] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:42.256511 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd57bbd1-5872-438a-aa1f-8647e53a24a8 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:42.257695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945c6ca-cd7c-4a14-b411-600fbbc83c4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:42.257841 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bd57bbd1-5872-438a-aa1f-8647e53a24a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:42.259414 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945c6ca-cd7c-4a14-b411-600fbbc83c4f service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:42.268833 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1360/2718] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:42.269504 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945c6ca-cd7c-4a14-b411-600fbbc83c4f service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:42.269636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd57bbd1-5872-438a-aa1f-8647e53a24a8 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:42.274815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8279f-7dcc-411c-841a-ee74d030c027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:42.275361 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8279f-7dcc-411c-841a-ee74d030c027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:42.275682 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8279f-7dcc-411c-841a-ee74d030c027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:42.275961 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8279f-7dcc-411c-841a-ee74d030c027 admin admin] PATH_INFO: `/v3/users/a575680063614186b324a7f93fef8d5c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:42.277113 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e8279f-7dcc-411c-841a-ee74d030c027 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a575680063614186b324a7f93fef8d5c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:42.279027 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e8279f-7dcc-411c-841a-ee74d030c027 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:42.279926 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945c6ca-cd7c-4a14-b411-600fbbc83c4f service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:42.287048 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc9848-05df-405e-ae73-c75642673a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:42.287699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc9848-05df-405e-ae73-c75642673a29 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:42.288047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc9848-05df-405e-ae73-c75642673a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:42.288245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc9848-05df-405e-ae73-c75642673a29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:42.288897 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc9848-05df-405e-ae73-c75642673a29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:16:42.289215 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc9848-05df-405e-ae73-c75642673a29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:42.290821 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abc9848-05df-405e-ae73-c75642673a29 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:42.307362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1361/2719] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:16:42.308557 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc9848-05df-405e-ae73-c75642673a29 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:42.321268 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc9848-05df-405e-ae73-c75642673a29 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:42.326125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d6d60-82d2-497d-bdd2-8a73573a006b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:42.326767 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d6d60-82d2-497d-bdd2-8a73573a006b admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:42.327115 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d6d60-82d2-497d-bdd2-8a73573a006b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:42.327400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d6d60-82d2-497d-bdd2-8a73573a006b admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:42.328073 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d6d60-82d2-497d-bdd2-8a73573a006b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:42.329996 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d6d60-82d2-497d-bdd2-8a73573a006b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:42.331877 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-41e8279f-7dcc-411c-841a-ee74d030c027 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a575680063614186b324a7f93fef8d5c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:42.332587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-41e8279f-7dcc-411c-841a-ee74d030c027 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a575680063614186b324a7f93fef8d5c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:42.340924 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-41e8279f-7dcc-411c-841a-ee74d030c027 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a575680063614186b324a7f93fef8d5c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:42.341852 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-41e8279f-7dcc-411c-841a-ee74d030c027 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a575680063614186b324a7f93fef8d5c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:42.351142 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-41e8279f-7dcc-411c-841a-ee74d030c027 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a575680063614186b324a7f93fef8d5c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:42.366585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1362/2720] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:42 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:16:42.368008 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d6d60-82d2-497d-bdd2-8a73573a006b admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:42.369890 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5a4d6d60-82d2-497d-bdd2-8a73573a006b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:42.371066 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1359/2721] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:42 2022] DELETE /identity/v3/users/a575680063614186b324a7f93fef8d5c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:42.447955 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d6d60-82d2-497d-bdd2-8a73573a006b admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:42.523215 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bcf158-b768-4f3b-bfe5-7b72625b9926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:42.524402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bcf158-b768-4f3b-bfe5-7b72625b9926 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:42.524697 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bcf158-b768-4f3b-bfe5-7b72625b9926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:42.525057 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bcf158-b768-4f3b-bfe5-7b72625b9926 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:42.525818 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bcf158-b768-4f3b-bfe5-7b72625b9926 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:42.529305 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bcf158-b768-4f3b-bfe5-7b72625b9926 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:42.578061 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1363/2722] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:42 2022] POST /identity/v3/users => generated 293 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:16:42.581934 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8279f-7dcc-411c-841a-ee74d030c027 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:42.593050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e8279f-7dcc-411c-841a-ee74d030c027 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:42.598249 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f1e05-618b-4616-8146-65e4ba8886b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:42.598867 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f1e05-618b-4616-8146-65e4ba8886b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:42.599203 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f1e05-618b-4616-8146-65e4ba8886b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:42.599482 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f1e05-618b-4616-8146-65e4ba8886b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:42.600176 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f1e05-618b-4616-8146-65e4ba8886b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:42.601988 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f1e05-618b-4616-8146-65e4ba8886b5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:42.625053 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1360/2723] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:16:42.629185 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bcf158-b768-4f3b-bfe5-7b72625b9926 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:42.640964 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bcf158-b768-4f3b-bfe5-7b72625b9926 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:42.646286 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514a7d6-5a44-47d1-a745-228fe37e5777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:42.646972 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514a7d6-5a44-47d1-a745-228fe37e5777 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:42.647257 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514a7d6-5a44-47d1-a745-228fe37e5777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:42.647529 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514a7d6-5a44-47d1-a745-228fe37e5777 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:42.648278 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514a7d6-5a44-47d1-a745-228fe37e5777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:42.650043 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514a7d6-5a44-47d1-a745-228fe37e5777 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:42.664155 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1364/2724] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:42.665426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f1e05-618b-4616-8146-65e4ba8886b5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:42.678550 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f1e05-618b-4616-8146-65e4ba8886b5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:42.683767 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b5b2e-7982-4a80-98e7-528918b58070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:42.684437 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b5b2e-7982-4a80-98e7-528918b58070 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:42.684718 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b5b2e-7982-4a80-98e7-528918b58070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:42.685035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b5b2e-7982-4a80-98e7-528918b58070 admin admin] PATH_INFO: `/v3/projects/cae35196ae584b5596bfbb358759c415/users/d65a35936a7b400ba1e9e99b45ecfec4/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:42.689389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514a7d6-5a44-47d1-a745-228fe37e5777 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:42.691221 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b514a7d6-5a44-47d1-a745-228fe37e5777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:42.705476 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b5b2e-7982-4a80-98e7-528918b58070 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cae35196ae584b5596bfbb358759c415, user_id=d65a35936a7b400ba1e9e99b45ecfec4, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:42.708387 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b5b2e-7982-4a80-98e7-528918b58070 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:42.715726 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514a7d6-5a44-47d1-a745-228fe37e5777 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:42.721260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f417d-12a1-47da-aca9-515965b93fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:42.721907 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f417d-12a1-47da-aca9-515965b93fe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:42.722195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f417d-12a1-47da-aca9-515965b93fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:42.722473 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f417d-12a1-47da-aca9-515965b93fe1 admin admin] PATH_INFO: `/v3/projects/42d42f56546d4ef69a258ffb5edccac1` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:42.723680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f417d-12a1-47da-aca9-515965b93fe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42d42f56546d4ef69a258ffb5edccac1)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:42.724302 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1361/2725] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:42 2022] PUT /identity/v3/projects/cae35196ae584b5596bfbb358759c415/users/d65a35936a7b400ba1e9e99b45ecfec4/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:42.725619 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f417d-12a1-47da-aca9-515965b93fe1 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:42.737189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65566e-017b-4748-8780-2b0792d08e78 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:42.737706 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65566e-017b-4748-8780-2b0792d08e78 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:42.737967 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65566e-017b-4748-8780-2b0792d08e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:42.754364 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:42.769343 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ae65566e-017b-4748-8780-2b0792d08e78 None None] MFA Rules not processed for user `d65a35936a7b400ba1e9e99b45ecfec4`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:42.770675 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ae65566e-017b-4748-8780-2b0792d08e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:42.779967 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ef8f417d-12a1-47da-aca9-515965b93fe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42d42f56546d4ef69a258ffb5edccac1', 'request_id': 'req-ef8f417d-12a1-47da-aca9-515965b93fe1'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:42.787625 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ef8f417d-12a1-47da-aca9-515965b93fe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42d42f56546d4ef69a258ffb5edccac1', 'request_id': 'req-ef8f417d-12a1-47da-aca9-515965b93fe1'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:42.793627 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ef8f417d-12a1-47da-aca9-515965b93fe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42d42f56546d4ef69a258ffb5edccac1', 'request_id': 'req-ef8f417d-12a1-47da-aca9-515965b93fe1'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:42.807095 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ef8f417d-12a1-47da-aca9-515965b93fe1 admin admin] The token cache is being invalidate because project 42d42f56546d4ef69a258ffb5edccac1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:42.807383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ef8f417d-12a1-47da-aca9-515965b93fe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:42.808086 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ef8f417d-12a1-47da-aca9-515965b93fe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:42.809552 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1365/2726] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:42 2022] DELETE /identity/v3/projects/42d42f56546d4ef69a258ffb5edccac1 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:16:42.815091 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5b565-4ee0-477e-81f0-bea618009ab0 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:42.815383 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5b565-4ee0-477e-81f0-bea618009ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:42.815688 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5b565-4ee0-477e-81f0-bea618009ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:42.816554 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:42.835342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d1b5b565-4ee0-477e-81f0-bea618009ab0 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:42.836584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d1b5b565-4ee0-477e-81f0-bea618009ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:42.838904 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1362/2727] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:16:42.851440 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65566e-017b-4748-8780-2b0792d08e78 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:42.852868 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ae65566e-017b-4748-8780-2b0792d08e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:42.903576 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1366/2728] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:16:42.910251 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65566e-017b-4748-8780-2b0792d08e78 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:42.911842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ca54c-32e2-4eb9-b12b-187be71917cf None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:42.912130 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ca54c-32e2-4eb9-b12b-187be71917cf None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:42.912402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ca54c-32e2-4eb9-b12b-187be71917cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:42.914394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b38651-4018-40a9-8a7b-9d7471ac8401 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:42.914999 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b38651-4018-40a9-8a7b-9d7471ac8401 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:42.915281 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b38651-4018-40a9-8a7b-9d7471ac8401 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:42.915577 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b38651-4018-40a9-8a7b-9d7471ac8401 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:42.924789 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b38651-4018-40a9-8a7b-9d7471ac8401 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:42.926218 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b38651-4018-40a9-8a7b-9d7471ac8401 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:42.932617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d80ca54c-32e2-4eb9-b12b-187be71917cf None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:42.933810 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d80ca54c-32e2-4eb9-b12b-187be71917cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:42.935265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1363/2729] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:16:42.942045 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1367/2730] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:16:42.946221 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b38651-4018-40a9-8a7b-9d7471ac8401 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:42.956311 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ca54c-32e2-4eb9-b12b-187be71917cf None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:42.956919 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b38651-4018-40a9-8a7b-9d7471ac8401 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:42.957723 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d80ca54c-32e2-4eb9-b12b-187be71917cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:42.961813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c64760-8161-4b0f-af81-67c02cd978fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:42.962446 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c64760-8161-4b0f-af81-67c02cd978fc admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:42.962730 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c64760-8161-4b0f-af81-67c02cd978fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:42.963011 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c64760-8161-4b0f-af81-67c02cd978fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:42.963724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c64760-8161-4b0f-af81-67c02cd978fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:16:42.964036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c64760-8161-4b0f-af81-67c02cd978fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:42.965747 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c64760-8161-4b0f-af81-67c02cd978fc admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:42.970277 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ca54c-32e2-4eb9-b12b-187be71917cf None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:42.975066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a41388-6cef-4d23-bc82-7baf1f17ff4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:42.975728 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a41388-6cef-4d23-bc82-7baf1f17ff4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:42.976007 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a41388-6cef-4d23-bc82-7baf1f17ff4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:42.976278 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a41388-6cef-4d23-bc82-7baf1f17ff4c admin admin] PATH_INFO: `/v3/users/d65a35936a7b400ba1e9e99b45ecfec4` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:42.977406 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a41388-6cef-4d23-bc82-7baf1f17ff4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d65a35936a7b400ba1e9e99b45ecfec4)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:42.981839 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1364/2731] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:16:42.981983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a41388-6cef-4d23-bc82-7baf1f17ff4c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:42.983244 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c64760-8161-4b0f-af81-67c02cd978fc admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:42.993561 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c64760-8161-4b0f-af81-67c02cd978fc admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:42.998223 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b98df9-d93c-4947-b79b-b2d647f9d5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:42.999320 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b98df9-d93c-4947-b79b-b2d647f9d5d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:42.999729 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b98df9-d93c-4947-b79b-b2d647f9d5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:43.000010 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b98df9-d93c-4947-b79b-b2d647f9d5d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:43.000636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b98df9-d93c-4947-b79b-b2d647f9d5d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:43.002516 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b98df9-d93c-4947-b79b-b2d647f9d5d6 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:43.028623 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d4a41388-6cef-4d23-bc82-7baf1f17ff4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd65a35936a7b400ba1e9e99b45ecfec4'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:43.029222 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d4a41388-6cef-4d23-bc82-7baf1f17ff4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd65a35936a7b400ba1e9e99b45ecfec4'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:43.031612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1365/2732] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:42 2022] POST /identity/v3/projects => generated 399 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:16:43.035782 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b98df9-d93c-4947-b79b-b2d647f9d5d6 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:43.038159 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d4a41388-6cef-4d23-bc82-7baf1f17ff4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd65a35936a7b400ba1e9e99b45ecfec4'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:43.045424 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d4a41388-6cef-4d23-bc82-7baf1f17ff4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd65a35936a7b400ba1e9e99b45ecfec4'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:43.046896 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d4a41388-6cef-4d23-bc82-7baf1f17ff4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd65a35936a7b400ba1e9e99b45ecfec4'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:43.064826 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1368/2733] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:42 2022] DELETE /identity/v3/users/d65a35936a7b400ba1e9e99b45ecfec4 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:16:43.069324 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfab8b9-f98e-47be-8f0c-06afe368f268 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:43.069791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfab8b9-f98e-47be-8f0c-06afe368f268 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:43.070072 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfab8b9-f98e-47be-8f0c-06afe368f268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:43.088810 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-abfab8b9-f98e-47be-8f0c-06afe368f268 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:43.090026 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-abfab8b9-f98e-47be-8f0c-06afe368f268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:43.099162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b98df9-d93c-4947-b79b-b2d647f9d5d6 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:43.102288 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:43.154373 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1369/2734] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:16:43.160188 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c79d3-a4a0-4be8-be9a-63aba4a3fec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:43.160814 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c79d3-a4a0-4be8-be9a-63aba4a3fec5 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:43.161097 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c79d3-a4a0-4be8-be9a-63aba4a3fec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:43.161373 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c79d3-a4a0-4be8-be9a-63aba4a3fec5 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:43.162006 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c79d3-a4a0-4be8-be9a-63aba4a3fec5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:43.164025 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c79d3-a4a0-4be8-be9a-63aba4a3fec5 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:43.165098 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfab8b9-f98e-47be-8f0c-06afe368f268 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:43.166468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-abfab8b9-f98e-47be-8f0c-06afe368f268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:43.174316 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:43.188701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1366/2735] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:43 2022] POST /identity/v3/users => generated 300 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:16:43.192890 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c79d3-a4a0-4be8-be9a-63aba4a3fec5 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:43.194262 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-848c79d3-a4a0-4be8-be9a-63aba4a3fec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:43.206231 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c79d3-a4a0-4be8-be9a-63aba4a3fec5 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:43.210789 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e56abf-086f-4b15-8e8c-0fcc05081996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:43.211384 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e56abf-086f-4b15-8e8c-0fcc05081996 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:43.211699 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e56abf-086f-4b15-8e8c-0fcc05081996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:43.211975 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e56abf-086f-4b15-8e8c-0fcc05081996 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:43.212610 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e56abf-086f-4b15-8e8c-0fcc05081996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:43.214228 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e56abf-086f-4b15-8e8c-0fcc05081996 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:43.227991 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1367/2736] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:16:43.229295 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e56abf-086f-4b15-8e8c-0fcc05081996 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:43.230755 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfab8b9-f98e-47be-8f0c-06afe368f268 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:43.234635 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e3380-de59-4eba-918b-7fd75110881c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:43.235298 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e3380-de59-4eba-918b-7fd75110881c service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:43.235977 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e3380-de59-4eba-918b-7fd75110881c service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:43.235977 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e3380-de59-4eba-918b-7fd75110881c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:43.240110 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e56abf-086f-4b15-8e8c-0fcc05081996 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:43.245933 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574cba1-9f49-4f20-9976-dc29975d7dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:43.246115 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e3380-de59-4eba-918b-7fd75110881c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:43.246637 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574cba1-9f49-4f20-9976-dc29975d7dec admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:43.246859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574cba1-9f49-4f20-9976-dc29975d7dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:43.247177 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574cba1-9f49-4f20-9976-dc29975d7dec admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:43.247469 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e3380-de59-4eba-918b-7fd75110881c service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:43.247884 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9574cba1-9f49-4f20-9976-dc29975d7dec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:43.249547 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9574cba1-9f49-4f20-9976-dc29975d7dec admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:43.257091 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1370/2737] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:16:43.263597 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1368/2738] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:43.266156 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e3380-de59-4eba-918b-7fd75110881c service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:43.276752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e3380-de59-4eba-918b-7fd75110881c service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:43.282154 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af274b1-58b9-48bc-8133-4de93630d1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:43.282757 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af274b1-58b9-48bc-8133-4de93630d1b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:43.283042 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af274b1-58b9-48bc-8133-4de93630d1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:43.283323 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af274b1-58b9-48bc-8133-4de93630d1b3 admin admin] PATH_INFO: `/v3/projects/3395e857ae504769946c2e290d631d0b/users/b8971ed4aae641b582c32ebf47dbe14e/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:43.299486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af274b1-58b9-48bc-8133-4de93630d1b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3395e857ae504769946c2e290d631d0b, user_id=b8971ed4aae641b582c32ebf47dbe14e, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:43.302398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af274b1-58b9-48bc-8133-4de93630d1b3 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:43.316430 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1371/2739] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:43 2022] PUT /identity/v3/projects/3395e857ae504769946c2e290d631d0b/users/b8971ed4aae641b582c32ebf47dbe14e/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:43.327511 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1250d-25e3-4986-a34b-355405273e79 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:43.327837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1250d-25e3-4986-a34b-355405273e79 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:43.328162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1250d-25e3-4986-a34b-355405273e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:43.358702 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d1c1250d-25e3-4986-a34b-355405273e79 None None] MFA Rules not processed for user `b8971ed4aae641b582c32ebf47dbe14e`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:43.360065 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d1c1250d-25e3-4986-a34b-355405273e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:43.435338 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1369/2740] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:43 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:43.439738 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af274b1-58b9-48bc-8133-4de93630d1b3 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:43.511331 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af274b1-58b9-48bc-8133-4de93630d1b3 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:43.516158 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162d049-4208-409e-a66a-977057b2a7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:43.516764 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162d049-4208-409e-a66a-977057b2a7b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:43.517049 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162d049-4208-409e-a66a-977057b2a7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:43.517326 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162d049-4208-409e-a66a-977057b2a7b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:43.517984 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d162d049-4208-409e-a66a-977057b2a7b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:43.520002 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d162d049-4208-409e-a66a-977057b2a7b2 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:43.550193 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1372/2741] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:43 2022] POST /identity/v3/projects => generated 401 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:16:43.554149 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1250d-25e3-4986-a34b-355405273e79 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:43.557459 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162d049-4208-409e-a66a-977057b2a7b2 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:43.559300 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d162d049-4208-409e-a66a-977057b2a7b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:43.574648 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1250d-25e3-4986-a34b-355405273e79 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:43.592317 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:43.638457 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162d049-4208-409e-a66a-977057b2a7b2 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:43.645049 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275e375-615e-434a-8bb3-96d8c1906336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:43.645808 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275e375-615e-434a-8bb3-96d8c1906336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:43.646258 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deee1fe-465f-4bfe-9609-3e29d16bbae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:43.646425 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275e375-615e-434a-8bb3-96d8c1906336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:43.646656 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275e375-615e-434a-8bb3-96d8c1906336 admin admin] PATH_INFO: `/v3/projects/cae35196ae584b5596bfbb358759c415` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:43.646870 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deee1fe-465f-4bfe-9609-3e29d16bbae2 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:43.647147 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deee1fe-465f-4bfe-9609-3e29d16bbae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:43.647517 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deee1fe-465f-4bfe-9609-3e29d16bbae2 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:43.648118 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a275e375-615e-434a-8bb3-96d8c1906336 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cae35196ae584b5596bfbb358759c415)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:43.648233 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deee1fe-465f-4bfe-9609-3e29d16bbae2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:43.650179 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deee1fe-465f-4bfe-9609-3e29d16bbae2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:43.651003 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a275e375-615e-434a-8bb3-96d8c1906336 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:43.677105 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1370/2742] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:43 2022] POST /identity/v3/users => generated 307 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:43.684766 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deee1fe-465f-4bfe-9609-3e29d16bbae2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:43.695791 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deee1fe-465f-4bfe-9609-3e29d16bbae2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:43.707594 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a275e375-615e-434a-8bb3-96d8c1906336 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cae35196ae584b5596bfbb358759c415', 'request_id': 'req-a275e375-615e-434a-8bb3-96d8c1906336'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:43.715659 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a275e375-615e-434a-8bb3-96d8c1906336 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cae35196ae584b5596bfbb358759c415', 'request_id': 'req-a275e375-615e-434a-8bb3-96d8c1906336'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:43.723484 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a275e375-615e-434a-8bb3-96d8c1906336 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cae35196ae584b5596bfbb358759c415', 'request_id': 'req-a275e375-615e-434a-8bb3-96d8c1906336'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:43.737729 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a275e375-615e-434a-8bb3-96d8c1906336 admin admin] The token cache is being invalidate because project cae35196ae584b5596bfbb358759c415 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:43.738012 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a275e375-615e-434a-8bb3-96d8c1906336 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:43.738677 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a275e375-615e-434a-8bb3-96d8c1906336 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:43.740864 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1373/2743] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:43 2022] DELETE /identity/v3/projects/cae35196ae584b5596bfbb358759c415 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:16:43.746867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f1a35-2d58-4789-b64c-bf78f37b6bdd None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:43.747165 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f1a35-2d58-4789-b64c-bf78f37b6bdd None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:43.747443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f1a35-2d58-4789-b64c-bf78f37b6bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:43.750756 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:43.769743 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-496f1a35-2d58-4789-b64c-bf78f37b6bdd None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:43.770951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-496f1a35-2d58-4789-b64c-bf78f37b6bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:43.772199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c62f3c-c66f-48a5-abb2-ef3685fe688f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:43.772815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c62f3c-c66f-48a5-abb2-ef3685fe688f admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:43.773098 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c62f3c-c66f-48a5-abb2-ef3685fe688f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:43.773428 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c62f3c-c66f-48a5-abb2-ef3685fe688f admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:43.774173 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c62f3c-c66f-48a5-abb2-ef3685fe688f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:43.775822 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c62f3c-c66f-48a5-abb2-ef3685fe688f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:43.781983 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1374/2744] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:43.789096 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1371/2745] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:43.789648 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debff30-5aa6-4fa0-b429-8ce0101ceebf None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:43.789993 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debff30-5aa6-4fa0-b429-8ce0101ceebf None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:43.790279 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debff30-5aa6-4fa0-b429-8ce0101ceebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:43.793175 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c62f3c-c66f-48a5-abb2-ef3685fe688f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:43.794962 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-92c62f3c-c66f-48a5-abb2-ef3685fe688f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:43.810724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c62f3c-c66f-48a5-abb2-ef3685fe688f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:43.810932 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-6debff30-5aa6-4fa0-b429-8ce0101ceebf None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:43.812608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6debff30-5aa6-4fa0-b429-8ce0101ceebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:43.815930 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c173aa3-e980-4f29-8602-5b0ae9b1f989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:43.816661 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c173aa3-e980-4f29-8602-5b0ae9b1f989 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:43.817089 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c173aa3-e980-4f29-8602-5b0ae9b1f989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:43.817292 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c173aa3-e980-4f29-8602-5b0ae9b1f989 admin admin] PATH_INFO: `/v3/projects/79598b71a8834c6799039890e584cf6a/users/2b9ebc0f53a6475aa41c34a7d780264f/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:43.825533 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1375/2746] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:16:43.826508 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debff30-5aa6-4fa0-b429-8ce0101ceebf None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:43.837347 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debff30-5aa6-4fa0-b429-8ce0101ceebf None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:43.837703 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c173aa3-e980-4f29-8602-5b0ae9b1f989 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79598b71a8834c6799039890e584cf6a, user_id=2b9ebc0f53a6475aa41c34a7d780264f, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:43.840418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c173aa3-e980-4f29-8602-5b0ae9b1f989 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:43.844437 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f6eae-8f11-474d-8267-1e29fd0a3ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:43.845053 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f6eae-8f11-474d-8267-1e29fd0a3ceb admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:43.845390 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f6eae-8f11-474d-8267-1e29fd0a3ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:43.845629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f6eae-8f11-474d-8267-1e29fd0a3ceb admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:43.846376 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f6eae-8f11-474d-8267-1e29fd0a3ceb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:16:43.846581 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f6eae-8f11-474d-8267-1e29fd0a3ceb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:43.848291 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f6eae-8f11-474d-8267-1e29fd0a3ceb admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:43.859245 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1372/2747] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:43 2022] PUT /identity/v3/projects/79598b71a8834c6799039890e584cf6a/users/2b9ebc0f53a6475aa41c34a7d780264f/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:43.863153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c173aa3-e980-4f29-8602-5b0ae9b1f989 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:43.863314 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1376/2748] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:16:43.868481 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f6eae-8f11-474d-8267-1e29fd0a3ceb admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:43.879755 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f6eae-8f11-474d-8267-1e29fd0a3ceb admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:43.937306 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d3ea9-5431-46f2-9840-6d1cae080620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:43.937919 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d3ea9-5431-46f2-9840-6d1cae080620 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:43.938203 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d3ea9-5431-46f2-9840-6d1cae080620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:43.938550 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d3ea9-5431-46f2-9840-6d1cae080620 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:43.939113 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d3ea9-5431-46f2-9840-6d1cae080620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:43.941066 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d3ea9-5431-46f2-9840-6d1cae080620 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:43.943471 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c173aa3-e980-4f29-8602-5b0ae9b1f989 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:43.948150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad7437-f020-4fc5-8b61-3ad61727ddcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:43.948764 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad7437-f020-4fc5-8b61-3ad61727ddcf admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:43.949035 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad7437-f020-4fc5-8b61-3ad61727ddcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:43.949305 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad7437-f020-4fc5-8b61-3ad61727ddcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:43.949943 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad7437-f020-4fc5-8b61-3ad61727ddcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:43.951528 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad7437-f020-4fc5-8b61-3ad61727ddcf admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:43.962433 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1373/2749] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:43.966780 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad7437-f020-4fc5-8b61-3ad61727ddcf admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:43.971451 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1377/2750] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:43 2022] POST /identity/v3/projects => generated 363 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:16:43.975242 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d3ea9-5431-46f2-9840-6d1cae080620 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:43.977666 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad7437-f020-4fc5-8b61-3ad61727ddcf admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:43.988117 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:44.040494 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554abac7-0d93-4b3e-9fef-80c11eba0ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:44.041210 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554abac7-0d93-4b3e-9fef-80c11eba0ed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:44.041489 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554abac7-0d93-4b3e-9fef-80c11eba0ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:44.041756 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554abac7-0d93-4b3e-9fef-80c11eba0ed7 admin admin] PATH_INFO: `/v3/projects/79598b71a8834c6799039890e584cf6a/users/2b9ebc0f53a6475aa41c34a7d780264f/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:44.043854 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554abac7-0d93-4b3e-9fef-80c11eba0ed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79598b71a8834c6799039890e584cf6a, user_id=2b9ebc0f53a6475aa41c34a7d780264f, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:44.045088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d3ea9-5431-46f2-9840-6d1cae080620 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:44.046775 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554abac7-0d93-4b3e-9fef-80c11eba0ed7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:44.050224 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534db573-9eb1-4ec6-9263-848aed6948d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:44.051153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534db573-9eb1-4ec6-9263-848aed6948d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:44.051479 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534db573-9eb1-4ec6-9263-848aed6948d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:44.051789 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534db573-9eb1-4ec6-9263-848aed6948d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:44.052446 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534db573-9eb1-4ec6-9263-848aed6948d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:44.054481 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534db573-9eb1-4ec6-9263-848aed6948d9 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:44.058323 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1374/2751] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:43 2022] PUT /identity/v3/projects/79598b71a8834c6799039890e584cf6a/users/2b9ebc0f53a6475aa41c34a7d780264f/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:16:44.083860 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1378/2752] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:43 2022] POST /identity/v3/users => generated 282 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:16:44.085553 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554abac7-0d93-4b3e-9fef-80c11eba0ed7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:44.128885 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8408d-f335-422c-a47c-d0d4b145ddaa None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:44.129338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8408d-f335-422c-a47c-d0d4b145ddaa None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:44.130023 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8408d-f335-422c-a47c-d0d4b145ddaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:44.148963 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:44.161562 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554abac7-0d93-4b3e-9fef-80c11eba0ed7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:44.163498 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a7f8408d-f335-422c-a47c-d0d4b145ddaa None None] MFA Rules not processed for user `2b9ebc0f53a6475aa41c34a7d780264f`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:44.164737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a7f8408d-f335-422c-a47c-d0d4b145ddaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:44.166818 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d542f8f-dcde-459b-947f-6f19a498bf5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:44.167432 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d542f8f-dcde-459b-947f-6f19a498bf5e admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:44.167781 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d542f8f-dcde-459b-947f-6f19a498bf5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:44.168097 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d542f8f-dcde-459b-947f-6f19a498bf5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:44.168745 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d542f8f-dcde-459b-947f-6f19a498bf5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:44.170333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d542f8f-dcde-459b-947f-6f19a498bf5e admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:44.180816 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1375/2753] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:44.184839 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d542f8f-dcde-459b-947f-6f19a498bf5e admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:44.194521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d542f8f-dcde-459b-947f-6f19a498bf5e admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:44.199204 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87a2b1-d7c3-4605-84c1-a719ee844ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:44.199817 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87a2b1-d7c3-4605-84c1-a719ee844ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:44.200400 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87a2b1-d7c3-4605-84c1-a719ee844ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:44.200400 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87a2b1-d7c3-4605-84c1-a719ee844ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:44.200987 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87a2b1-d7c3-4605-84c1-a719ee844ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:44.203487 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87a2b1-d7c3-4605-84c1-a719ee844ea3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:44.213814 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1376/2754] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:16:44.218814 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87a2b1-d7c3-4605-84c1-a719ee844ea3 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:44.228370 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87a2b1-d7c3-4605-84c1-a719ee844ea3 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:44.233255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c692b0-0a6f-46c5-bde6-70454ca62f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:44.233907 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c692b0-0a6f-46c5-bde6-70454ca62f45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:44.234205 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c692b0-0a6f-46c5-bde6-70454ca62f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:44.234477 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c692b0-0a6f-46c5-bde6-70454ca62f45 admin admin] PATH_INFO: `/v3/projects/b2f19da78ef54565bf618add8926371e/users/1740d3d1f6df488ea071b2d8898e548c/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:44.244129 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1379/2755] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:44 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:16:44.250113 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c692b0-0a6f-46c5-bde6-70454ca62f45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2f19da78ef54565bf618add8926371e, user_id=1740d3d1f6df488ea071b2d8898e548c, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:44.252806 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c692b0-0a6f-46c5-bde6-70454ca62f45 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:44.255201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8408d-f335-422c-a47c-d0d4b145ddaa None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:44.256831 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a7f8408d-f335-422c-a47c-d0d4b145ddaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:44.267197 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1377/2756] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:44 2022] PUT /identity/v3/projects/b2f19da78ef54565bf618add8926371e/users/1740d3d1f6df488ea071b2d8898e548c/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:44.276859 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3757c85-8fa8-4365-9c0c-6a5ce7c455ab None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:44.277150 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3757c85-8fa8-4365-9c0c-6a5ce7c455ab None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:44.277430 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3757c85-8fa8-4365-9c0c-6a5ce7c455ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:44.304903 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-a3757c85-8fa8-4365-9c0c-6a5ce7c455ab None None] MFA Rules not processed for user `1740d3d1f6df488ea071b2d8898e548c`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:44.306118 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a3757c85-8fa8-4365-9c0c-6a5ce7c455ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:44.318452 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8408d-f335-422c-a47c-d0d4b145ddaa None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:44.360749 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:44.370653 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd840db-a99d-4fbc-9fde-23eb75380ac7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:44.371269 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd840db-a99d-4fbc-9fde-23eb75380ac7 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:44.371584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd840db-a99d-4fbc-9fde-23eb75380ac7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:44.371862 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd840db-a99d-4fbc-9fde-23eb75380ac7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:44.382037 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1378/2757] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:16:44.395898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3757c85-8fa8-4365-9c0c-6a5ce7c455ab None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:44.397949 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-a3757c85-8fa8-4365-9c0c-6a5ce7c455ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:44.411373 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3757c85-8fa8-4365-9c0c-6a5ce7c455ab None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:44.415025 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8186db62-37cc-4952-8293-614e6ccd2c9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:44.415665 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186db62-37cc-4952-8293-614e6ccd2c9d service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:44.415951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186db62-37cc-4952-8293-614e6ccd2c9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:44.416218 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186db62-37cc-4952-8293-614e6ccd2c9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:44.424665 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8186db62-37cc-4952-8293-614e6ccd2c9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:44.426085 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8186db62-37cc-4952-8293-614e6ccd2c9d service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:44.434901 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1379/2758] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:16:44.439343 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd840db-a99d-4fbc-9fde-23eb75380ac7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:44.441047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd840db-a99d-4fbc-9fde-23eb75380ac7 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:44.451329 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1380/2759] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:44 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:16:44.534001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8186db62-37cc-4952-8293-614e6ccd2c9d service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:44.542926 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8186db62-37cc-4952-8293-614e6ccd2c9d service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:44.546782 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c20d0-252b-49d2-b631-0b7ce1bfb2a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:44.547390 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c20d0-252b-49d2-b631-0b7ce1bfb2a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:44.547685 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c20d0-252b-49d2-b631-0b7ce1bfb2a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:44.547964 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c20d0-252b-49d2-b631-0b7ce1bfb2a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:44.549723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-775c20d0-252b-49d2-b631-0b7ce1bfb2a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:44.613512 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c20d0-252b-49d2-b631-0b7ce1bfb2a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:44.614980 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c20d0-252b-49d2-b631-0b7ce1bfb2a5 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:44.625505 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1380/2760] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:44 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:44.917228 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd840db-a99d-4fbc-9fde-23eb75380ac7 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:44.918760 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-5cd840db-a99d-4fbc-9fde-23eb75380ac7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:44.996683 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd840db-a99d-4fbc-9fde-23eb75380ac7 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:45.002213 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5979c1-0c5f-45a9-9be1-56485df30746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:45.002853 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5979c1-0c5f-45a9-9be1-56485df30746 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:45.003196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5979c1-0c5f-45a9-9be1-56485df30746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:45.003474 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5979c1-0c5f-45a9-9be1-56485df30746 admin admin] PATH_INFO: `/v3/users/05c420254b904aa9b7ff5b843293cbb0` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:45.004635 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5979c1-0c5f-45a9-9be1-56485df30746 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05c420254b904aa9b7ff5b843293cbb0)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:45.006860 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5979c1-0c5f-45a9-9be1-56485df30746 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:45.052644 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-aa5979c1-0c5f-45a9-9be1-56485df30746 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05c420254b904aa9b7ff5b843293cbb0'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:45.053290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-aa5979c1-0c5f-45a9-9be1-56485df30746 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05c420254b904aa9b7ff5b843293cbb0'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:45.060106 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-aa5979c1-0c5f-45a9-9be1-56485df30746 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05c420254b904aa9b7ff5b843293cbb0'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:45.067547 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-aa5979c1-0c5f-45a9-9be1-56485df30746 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05c420254b904aa9b7ff5b843293cbb0'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:45.068338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-aa5979c1-0c5f-45a9-9be1-56485df30746 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05c420254b904aa9b7ff5b843293cbb0'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:45.088956 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1381/2761] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:44 2022] DELETE /identity/v3/users/05c420254b904aa9b7ff5b843293cbb0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:45.094367 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9a962-eeaa-4848-a826-e5b79d9987ec None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:45.094836 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9a962-eeaa-4848-a826-e5b79d9987ec None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:45.094957 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9a962-eeaa-4848-a826-e5b79d9987ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:45.114015 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-3cc9a962-eeaa-4848-a826-e5b79d9987ec None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:45.115185 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3cc9a962-eeaa-4848-a826-e5b79d9987ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:45.134652 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:45.174060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1381/2762] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:16:45.183812 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5979c1-0c5f-45a9-9be1-56485df30746 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:45.185247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-aa5979c1-0c5f-45a9-9be1-56485df30746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:45.199078 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:45.248130 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5979c1-0c5f-45a9-9be1-56485df30746 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:45.251972 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e7394-a390-46d4-b2ac-56b846cfbe69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:45.252580 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e7394-a390-46d4-b2ac-56b846cfbe69 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:45.252854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e7394-a390-46d4-b2ac-56b846cfbe69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:45.253128 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e7394-a390-46d4-b2ac-56b846cfbe69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:45.262407 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e7394-a390-46d4-b2ac-56b846cfbe69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:45.263888 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e7394-a390-46d4-b2ac-56b846cfbe69 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:45.273686 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1382/2763] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:45.547243 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9a962-eeaa-4848-a826-e5b79d9987ec None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:45.548844 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3cc9a962-eeaa-4848-a826-e5b79d9987ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:45.561110 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9a962-eeaa-4848-a826-e5b79d9987ec None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:45.566620 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22a1d2-6be3-4663-bb42-f9197fc2f0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:45.567667 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22a1d2-6be3-4663-bb42-f9197fc2f0ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:45.567979 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22a1d2-6be3-4663-bb42-f9197fc2f0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:45.568254 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22a1d2-6be3-4663-bb42-f9197fc2f0ea admin admin] PATH_INFO: `/v3/projects/17c76b2507f74d758561d5d8a91322f4` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:45.570048 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd22a1d2-6be3-4663-bb42-f9197fc2f0ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=17c76b2507f74d758561d5d8a91322f4)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:45.571990 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd22a1d2-6be3-4663-bb42-f9197fc2f0ea admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:45.619326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fd22a1d2-6be3-4663-bb42-f9197fc2f0ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17c76b2507f74d758561d5d8a91322f4', 'request_id': 'req-fd22a1d2-6be3-4663-bb42-f9197fc2f0ea'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:45.626166 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fd22a1d2-6be3-4663-bb42-f9197fc2f0ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17c76b2507f74d758561d5d8a91322f4', 'request_id': 'req-fd22a1d2-6be3-4663-bb42-f9197fc2f0ea'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:45.632046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fd22a1d2-6be3-4663-bb42-f9197fc2f0ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17c76b2507f74d758561d5d8a91322f4', 'request_id': 'req-fd22a1d2-6be3-4663-bb42-f9197fc2f0ea'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:45.643600 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fd22a1d2-6be3-4663-bb42-f9197fc2f0ea admin admin] The token cache is being invalidate because project 17c76b2507f74d758561d5d8a91322f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:45.643893 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fd22a1d2-6be3-4663-bb42-f9197fc2f0ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:45.644530 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-fd22a1d2-6be3-4663-bb42-f9197fc2f0ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:45.645995 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1382/2764] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:45 2022] DELETE /identity/v3/projects/17c76b2507f74d758561d5d8a91322f4 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:48.784516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e7394-a390-46d4-b2ac-56b846cfbe69 service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:48.785999 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-021e7394-a390-46d4-b2ac-56b846cfbe69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:48.798545 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e7394-a390-46d4-b2ac-56b846cfbe69 service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:48.856752 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7313fafc-d683-496a-88c9-e4e01b5d7922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:48.857458 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313fafc-d683-496a-88c9-e4e01b5d7922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:48.857744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313fafc-d683-496a-88c9-e4e01b5d7922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:48.858028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313fafc-d683-496a-88c9-e4e01b5d7922 admin admin] PATH_INFO: `/v3/users/b8971ed4aae641b582c32ebf47dbe14e` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:48.859382 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7313fafc-d683-496a-88c9-e4e01b5d7922 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8971ed4aae641b582c32ebf47dbe14e)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:48.861450 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7313fafc-d683-496a-88c9-e4e01b5d7922 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:48.908633 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7313fafc-d683-496a-88c9-e4e01b5d7922 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8971ed4aae641b582c32ebf47dbe14e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:48.909339 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7313fafc-d683-496a-88c9-e4e01b5d7922 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8971ed4aae641b582c32ebf47dbe14e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:48.916100 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7313fafc-d683-496a-88c9-e4e01b5d7922 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8971ed4aae641b582c32ebf47dbe14e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:48.924035 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7313fafc-d683-496a-88c9-e4e01b5d7922 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8971ed4aae641b582c32ebf47dbe14e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:48.924673 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-7313fafc-d683-496a-88c9-e4e01b5d7922 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8971ed4aae641b582c32ebf47dbe14e'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:48.942667 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1383/2765] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:48 2022] DELETE /identity/v3/users/b8971ed4aae641b582c32ebf47dbe14e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:48.947152 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd561f69-d9f0-4594-b3ad-b702f6d8daef None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:48.947662 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd561f69-d9f0-4594-b3ad-b702f6d8daef None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:48.947774 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd561f69-d9f0-4594-b3ad-b702f6d8daef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:48.967885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-dd561f69-d9f0-4594-b3ad-b702f6d8daef None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:48.969296 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dd561f69-d9f0-4594-b3ad-b702f6d8daef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:49.028287 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1383/2766] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:49.038306 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7313fafc-d683-496a-88c9-e4e01b5d7922 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:49.039786 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7313fafc-d683-496a-88c9-e4e01b5d7922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:49.075459 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:49.099168 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7313fafc-d683-496a-88c9-e4e01b5d7922 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:49.102787 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016ca66-298d-4fd7-b00b-cbd0c2b38a9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:49.103391 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016ca66-298d-4fd7-b00b-cbd0c2b38a9c service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:49.103708 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016ca66-298d-4fd7-b00b-cbd0c2b38a9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:49.103984 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016ca66-298d-4fd7-b00b-cbd0c2b38a9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:49.112842 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016ca66-298d-4fd7-b00b-cbd0c2b38a9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:49.114303 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016ca66-298d-4fd7-b00b-cbd0c2b38a9c service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:49.123693 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1384/2767] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:49.393752 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd561f69-d9f0-4594-b3ad-b702f6d8daef None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:49.395173 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dd561f69-d9f0-4594-b3ad-b702f6d8daef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:49.408615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd561f69-d9f0-4594-b3ad-b702f6d8daef None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:49.413602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f8265-669f-4f7e-a015-1eed410bbe2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:49.414256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f8265-669f-4f7e-a015-1eed410bbe2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:49.414704 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f8265-669f-4f7e-a015-1eed410bbe2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:49.415013 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f8265-669f-4f7e-a015-1eed410bbe2d admin admin] PATH_INFO: `/v3/projects/3395e857ae504769946c2e290d631d0b` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:49.416215 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f8265-669f-4f7e-a015-1eed410bbe2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3395e857ae504769946c2e290d631d0b)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:49.418163 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f8265-669f-4f7e-a015-1eed410bbe2d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:49.462341 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:49.465572 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7b0f8265-669f-4f7e-a015-1eed410bbe2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3395e857ae504769946c2e290d631d0b', 'request_id': 'req-7b0f8265-669f-4f7e-a015-1eed410bbe2d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:49.474495 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7b0f8265-669f-4f7e-a015-1eed410bbe2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3395e857ae504769946c2e290d631d0b', 'request_id': 'req-7b0f8265-669f-4f7e-a015-1eed410bbe2d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:49.481033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7b0f8265-669f-4f7e-a015-1eed410bbe2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3395e857ae504769946c2e290d631d0b', 'request_id': 'req-7b0f8265-669f-4f7e-a015-1eed410bbe2d'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:49.492964 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7b0f8265-669f-4f7e-a015-1eed410bbe2d admin admin] The token cache is being invalidate because project 3395e857ae504769946c2e290d631d0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:49.493252 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7b0f8265-669f-4f7e-a015-1eed410bbe2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:49.493902 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-7b0f8265-669f-4f7e-a015-1eed410bbe2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:49.495408 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1384/2768] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:49 2022] DELETE /identity/v3/projects/3395e857ae504769946c2e290d631d0b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:49.500424 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016ca66-298d-4fd7-b00b-cbd0c2b38a9c service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:49.510168 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016ca66-298d-4fd7-b00b-cbd0c2b38a9c service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:49.565964 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab763e36-6130-47e5-840e-5814fcbff198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:49.566516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab763e36-6130-47e5-840e-5814fcbff198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:49.566799 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab763e36-6130-47e5-840e-5814fcbff198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:49.567077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab763e36-6130-47e5-840e-5814fcbff198 admin admin] PATH_INFO: `/v3/users/2b9ebc0f53a6475aa41c34a7d780264f` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:49.568207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab763e36-6130-47e5-840e-5814fcbff198 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b9ebc0f53a6475aa41c34a7d780264f)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:49.570130 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab763e36-6130-47e5-840e-5814fcbff198 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:49.615654 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ab763e36-6130-47e5-840e-5814fcbff198 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b9ebc0f53a6475aa41c34a7d780264f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:49.616324 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ab763e36-6130-47e5-840e-5814fcbff198 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b9ebc0f53a6475aa41c34a7d780264f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:49.623483 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ab763e36-6130-47e5-840e-5814fcbff198 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b9ebc0f53a6475aa41c34a7d780264f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:49.630438 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ab763e36-6130-47e5-840e-5814fcbff198 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b9ebc0f53a6475aa41c34a7d780264f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:49.631069 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-ab763e36-6130-47e5-840e-5814fcbff198 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b9ebc0f53a6475aa41c34a7d780264f'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:49.648508 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1385/2769] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:49 2022] DELETE /identity/v3/users/2b9ebc0f53a6475aa41c34a7d780264f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:16:49.929731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f8265-669f-4f7e-a015-1eed410bbe2d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:49.931413 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7b0f8265-669f-4f7e-a015-1eed410bbe2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:49.948926 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f8265-669f-4f7e-a015-1eed410bbe2d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:50.038530 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06be7b-fabb-42e2-a8ca-2ecea5c6c6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:50.039167 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06be7b-fabb-42e2-a8ca-2ecea5c6c6eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:50.039439 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06be7b-fabb-42e2-a8ca-2ecea5c6c6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:50.039817 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06be7b-fabb-42e2-a8ca-2ecea5c6c6eb admin admin] PATH_INFO: `/v3/projects/79598b71a8834c6799039890e584cf6a` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:50.040983 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df06be7b-fabb-42e2-a8ca-2ecea5c6c6eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=79598b71a8834c6799039890e584cf6a)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:50.042883 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df06be7b-fabb-42e2-a8ca-2ecea5c6c6eb admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:50.089620 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-df06be7b-fabb-42e2-a8ca-2ecea5c6c6eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79598b71a8834c6799039890e584cf6a', 'request_id': 'req-df06be7b-fabb-42e2-a8ca-2ecea5c6c6eb'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:50.097215 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-df06be7b-fabb-42e2-a8ca-2ecea5c6c6eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79598b71a8834c6799039890e584cf6a', 'request_id': 'req-df06be7b-fabb-42e2-a8ca-2ecea5c6c6eb'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:50.102821 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-df06be7b-fabb-42e2-a8ca-2ecea5c6c6eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79598b71a8834c6799039890e584cf6a', 'request_id': 'req-df06be7b-fabb-42e2-a8ca-2ecea5c6c6eb'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:50.114226 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-df06be7b-fabb-42e2-a8ca-2ecea5c6c6eb admin admin] The token cache is being invalidate because project 79598b71a8834c6799039890e584cf6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:50.114510 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-df06be7b-fabb-42e2-a8ca-2ecea5c6c6eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:50.115135 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-df06be7b-fabb-42e2-a8ca-2ecea5c6c6eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:50.116617 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1385/2770] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:49 2022] DELETE /identity/v3/projects/79598b71a8834c6799039890e584cf6a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:50.121496 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b4a1a-ffcc-4df2-b19c-eded0725433e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:50.121987 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b4a1a-ffcc-4df2-b19c-eded0725433e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:50.122212 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b4a1a-ffcc-4df2-b19c-eded0725433e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:50.142505 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-0b3b4a1a-ffcc-4df2-b19c-eded0725433e None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:50.143746 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-0b3b4a1a-ffcc-4df2-b19c-eded0725433e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:50.194201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:50.207024 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1386/2771] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:50.214847 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585d55a-70bf-45c7-8ea5-5e88a21d7a77 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:50.215131 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585d55a-70bf-45c7-8ea5-5e88a21d7a77 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:50.215402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585d55a-70bf-45c7-8ea5-5e88a21d7a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:50.235580 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-c585d55a-70bf-45c7-8ea5-5e88a21d7a77 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:50.236793 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-c585d55a-70bf-45c7-8ea5-5e88a21d7a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:50.245492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1386/2772] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:50.250427 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b4a1a-ffcc-4df2-b19c-eded0725433e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:50.260397 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b4a1a-ffcc-4df2-b19c-eded0725433e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:50.265431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d115568-e97f-4991-a061-f5f26e7142e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:50.266035 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d115568-e97f-4991-a061-f5f26e7142e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:50.266316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d115568-e97f-4991-a061-f5f26e7142e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:50.266588 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d115568-e97f-4991-a061-f5f26e7142e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:50.267230 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d115568-e97f-4991-a061-f5f26e7142e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:16:50.267534 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d115568-e97f-4991-a061-f5f26e7142e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:50.269256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d115568-e97f-4991-a061-f5f26e7142e7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:50.280822 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1387/2773] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:16:50.285847 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585d55a-70bf-45c7-8ea5-5e88a21d7a77 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:50.295343 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585d55a-70bf-45c7-8ea5-5e88a21d7a77 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:50.300783 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2eec4-2b81-4d26-9991-43efd467bc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:50.301378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2eec4-2b81-4d26-9991-43efd467bc3d admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:50.301665 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2eec4-2b81-4d26-9991-43efd467bc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:50.301938 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2eec4-2b81-4d26-9991-43efd467bc3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:50.302569 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e2eec4-2b81-4d26-9991-43efd467bc3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:50.304499 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e2eec4-2b81-4d26-9991-43efd467bc3d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:50.331690 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1387/2774] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:50 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:50.335897 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d115568-e97f-4991-a061-f5f26e7142e7 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:50.346439 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d115568-e97f-4991-a061-f5f26e7142e7 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:50.402074 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2df6e-2334-4cf4-9e42-b273a719f8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:50.402677 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2df6e-2334-4cf4-9e42-b273a719f8bb admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:50.402960 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2df6e-2334-4cf4-9e42-b273a719f8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:50.403236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2df6e-2334-4cf4-9e42-b273a719f8bb admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:50.403899 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb2df6e-2334-4cf4-9e42-b273a719f8bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:50.406382 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb2df6e-2334-4cf4-9e42-b273a719f8bb admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:50.427167 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1388/2775] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:50 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:16:50.431253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2eec4-2b81-4d26-9991-43efd467bc3d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:50.441436 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2eec4-2b81-4d26-9991-43efd467bc3d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:50.445939 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579d1b9-28c0-4217-9a28-7aa45539c773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:50.446532 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579d1b9-28c0-4217-9a28-7aa45539c773 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:50.447050 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579d1b9-28c0-4217-9a28-7aa45539c773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:50.447334 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579d1b9-28c0-4217-9a28-7aa45539c773 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:50.448009 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1579d1b9-28c0-4217-9a28-7aa45539c773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:50.449625 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1579d1b9-28c0-4217-9a28-7aa45539c773 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:50.459638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1388/2776] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:50.463353 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2df6e-2334-4cf4-9e42-b273a719f8bb admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:50.487737 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2df6e-2334-4cf4-9e42-b273a719f8bb admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:50.493764 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f47385-7862-447d-8f0f-ae6c9d2cba00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:50.494466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f47385-7862-447d-8f0f-ae6c9d2cba00 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:50.494821 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f47385-7862-447d-8f0f-ae6c9d2cba00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:50.495167 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f47385-7862-447d-8f0f-ae6c9d2cba00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:50.495918 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f47385-7862-447d-8f0f-ae6c9d2cba00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:50.497529 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f47385-7862-447d-8f0f-ae6c9d2cba00 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:50.509034 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1389/2777] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:50.513760 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579d1b9-28c0-4217-9a28-7aa45539c773 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:50.523765 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579d1b9-28c0-4217-9a28-7aa45539c773 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:50.528433 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c60427-b0aa-48d8-96c1-b18ab90129cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:50.529084 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c60427-b0aa-48d8-96c1-b18ab90129cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:50.529362 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c60427-b0aa-48d8-96c1-b18ab90129cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:50.529636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c60427-b0aa-48d8-96c1-b18ab90129cf admin admin] PATH_INFO: `/v3/projects/40a7a0aa6c7845ef994d0a4458c85479/users/e139f7b0acd140ec9618df2b129efb08/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:50.546541 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c60427-b0aa-48d8-96c1-b18ab90129cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=40a7a0aa6c7845ef994d0a4458c85479, user_id=e139f7b0acd140ec9618df2b129efb08, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:50.549396 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c60427-b0aa-48d8-96c1-b18ab90129cf admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:50.559054 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:50.565424 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1389/2778] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:50 2022] PUT /identity/v3/projects/40a7a0aa6c7845ef994d0a4458c85479/users/e139f7b0acd140ec9618df2b129efb08/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:50.576446 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd02644-21a7-4ef9-ad28-9d293206557a None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:50.576729 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd02644-21a7-4ef9-ad28-9d293206557a None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:50.577047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd02644-21a7-4ef9-ad28-9d293206557a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:50.608263 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-3cd02644-21a7-4ef9-ad28-9d293206557a None None] MFA Rules not processed for user `e139f7b0acd140ec9618df2b129efb08`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:50.610274 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-3cd02644-21a7-4ef9-ad28-9d293206557a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:50.675764 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1390/2779] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:16:50.680213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c60427-b0aa-48d8-96c1-b18ab90129cf admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:50.741811 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c60427-b0aa-48d8-96c1-b18ab90129cf admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:50.746879 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9374179-2714-4858-9a69-51df8e533219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:50.747536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9374179-2714-4858-9a69-51df8e533219 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:50.747852 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9374179-2714-4858-9a69-51df8e533219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:50.748130 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9374179-2714-4858-9a69-51df8e533219 admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:50.748778 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9374179-2714-4858-9a69-51df8e533219 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:50.750701 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9374179-2714-4858-9a69-51df8e533219 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:50.785709 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1390/2780] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:50 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:50.787380 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd02644-21a7-4ef9-ad28-9d293206557a None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:50.798606 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd02644-21a7-4ef9-ad28-9d293206557a None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:50.847274 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:50.857766 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfb68e-da2c-4782-9bbc-6e548e2370a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:50.858495 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfb68e-da2c-4782-9bbc-6e548e2370a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:50.858706 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfb68e-da2c-4782-9bbc-6e548e2370a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:50.858980 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfb68e-da2c-4782-9bbc-6e548e2370a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:50.859629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cfb68e-da2c-4782-9bbc-6e548e2370a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:50.861642 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cfb68e-da2c-4782-9bbc-6e548e2370a6 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:50.884382 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1391/2781] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:50 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:50.888519 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9374179-2714-4858-9a69-51df8e533219 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:50.899835 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9374179-2714-4858-9a69-51df8e533219 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:50.904392 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88722906-d043-4020-a349-d6aff0a9eebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:50.904980 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88722906-d043-4020-a349-d6aff0a9eebd admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:50.905253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88722906-d043-4020-a349-d6aff0a9eebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:50.906187 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88722906-d043-4020-a349-d6aff0a9eebd admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:50.906817 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88722906-d043-4020-a349-d6aff0a9eebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:50.908434 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88722906-d043-4020-a349-d6aff0a9eebd admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:50.918782 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1391/2782] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:50.922936 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfb68e-da2c-4782-9bbc-6e548e2370a6 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:50.932938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfb68e-da2c-4782-9bbc-6e548e2370a6 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:50.937628 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81938902-15b6-4b9a-b98c-8d660f5a21cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:50.938275 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81938902-15b6-4b9a-b98c-8d660f5a21cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:50.938557 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81938902-15b6-4b9a-b98c-8d660f5a21cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:50.938833 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81938902-15b6-4b9a-b98c-8d660f5a21cb admin admin] PATH_INFO: `/v3/projects/06bc322a172c4f1ea8d5f05ee1667db5/users/d5678b18cca4483bbb2f89f767386e08/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:51.076260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81938902-15b6-4b9a-b98c-8d660f5a21cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=06bc322a172c4f1ea8d5f05ee1667db5, user_id=d5678b18cca4483bbb2f89f767386e08, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:51.079519 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81938902-15b6-4b9a-b98c-8d660f5a21cb admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:51.093654 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1392/2783] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:50 2022] PUT /identity/v3/projects/06bc322a172c4f1ea8d5f05ee1667db5/users/d5678b18cca4483bbb2f89f767386e08/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:51.098134 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88722906-d043-4020-a349-d6aff0a9eebd admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:51.109209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88722906-d043-4020-a349-d6aff0a9eebd admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:51.167416 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4eb6a9-8b85-4eb4-bf3b-9edcaf1168f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:51.168059 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eb6a9-8b85-4eb4-bf3b-9edcaf1168f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:51.168343 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eb6a9-8b85-4eb4-bf3b-9edcaf1168f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:51.168618 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eb6a9-8b85-4eb4-bf3b-9edcaf1168f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:51.169257 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4eb6a9-8b85-4eb4-bf3b-9edcaf1168f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:51.170847 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4eb6a9-8b85-4eb4-bf3b-9edcaf1168f0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:51.180784 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1392/2784] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:16:51.185671 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81938902-15b6-4b9a-b98c-8d660f5a21cb admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:51.197778 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81938902-15b6-4b9a-b98c-8d660f5a21cb admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:51.204354 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03403961-5c9d-4108-9d77-65727f080b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:51.205009 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03403961-5c9d-4108-9d77-65727f080b63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:51.205308 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03403961-5c9d-4108-9d77-65727f080b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:51.205568 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03403961-5c9d-4108-9d77-65727f080b63 admin admin] PATH_INFO: `/v3/projects/06bc322a172c4f1ea8d5f05ee1667db5/users/d5678b18cca4483bbb2f89f767386e08/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:51.207964 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03403961-5c9d-4108-9d77-65727f080b63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06bc322a172c4f1ea8d5f05ee1667db5, user_id=d5678b18cca4483bbb2f89f767386e08, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:51.210247 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03403961-5c9d-4108-9d77-65727f080b63 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:51.220733 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1393/2785] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:51 2022] PUT /identity/v3/projects/06bc322a172c4f1ea8d5f05ee1667db5/users/d5678b18cca4483bbb2f89f767386e08/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:51.233208 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4ecd3-1de0-434a-ab34-a720f9d8309a None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:51.234116 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4ecd3-1de0-434a-ab34-a720f9d8309a None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:51.234425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4ecd3-1de0-434a-ab34-a720f9d8309a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:51.266354 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-39b4ecd3-1de0-434a-ab34-a720f9d8309a None None] MFA Rules not processed for user `d5678b18cca4483bbb2f89f767386e08`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:51.267907 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-39b4ecd3-1de0-434a-ab34-a720f9d8309a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:51.334700 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:51.345148 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1393/2786] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:51 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:16:51.361167 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03403961-5c9d-4108-9d77-65727f080b63 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:51.363769 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-03403961-5c9d-4108-9d77-65727f080b63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:51.429575 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03403961-5c9d-4108-9d77-65727f080b63 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:51.434390 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd55805-ecf0-4542-b09f-07ecf181607b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:51.435002 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd55805-ecf0-4542-b09f-07ecf181607b service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:51.435283 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd55805-ecf0-4542-b09f-07ecf181607b service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:51.435575 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd55805-ecf0-4542-b09f-07ecf181607b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:51.445338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd55805-ecf0-4542-b09f-07ecf181607b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:51.447009 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd55805-ecf0-4542-b09f-07ecf181607b service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:51.459281 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1394/2787] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:51 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:51.846711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4ecd3-1de0-434a-ab34-a720f9d8309a None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:51.911464 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4ecd3-1de0-434a-ab34-a720f9d8309a None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:51.916025 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51735bd-3739-4d0c-945d-59a498cff137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:51.916662 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51735bd-3739-4d0c-945d-59a498cff137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:51.916942 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51735bd-3739-4d0c-945d-59a498cff137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:51.917215 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51735bd-3739-4d0c-945d-59a498cff137 admin admin] PATH_INFO: `/v3/users/e139f7b0acd140ec9618df2b129efb08` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:51.918394 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51735bd-3739-4d0c-945d-59a498cff137 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e139f7b0acd140ec9618df2b129efb08)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:51.920350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51735bd-3739-4d0c-945d-59a498cff137 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:51.973639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e51735bd-3739-4d0c-945d-59a498cff137 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e139f7b0acd140ec9618df2b129efb08'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:51.974432 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e51735bd-3739-4d0c-945d-59a498cff137 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e139f7b0acd140ec9618df2b129efb08'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:51.988942 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e51735bd-3739-4d0c-945d-59a498cff137 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e139f7b0acd140ec9618df2b129efb08'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:51.990529 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd55805-ecf0-4542-b09f-07ecf181607b service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:51.991132 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e51735bd-3739-4d0c-945d-59a498cff137 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e139f7b0acd140ec9618df2b129efb08'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:51.993003 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9fd55805-ecf0-4542-b09f-07ecf181607b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:52.011390 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-e51735bd-3739-4d0c-945d-59a498cff137 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e139f7b0acd140ec9618df2b129efb08'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:52.041898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1394/2788] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:51 2022] DELETE /identity/v3/users/e139f7b0acd140ec9618df2b129efb08 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:52.046763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5fb66-e98e-41b4-bc1a-96ab6c5d9c2c None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:52.047053 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5fb66-e98e-41b4-bc1a-96ab6c5d9c2c None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:52.047365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5fb66-e98e-41b4-bc1a-96ab6c5d9c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:52.069342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-42f5fb66-e98e-41b4-bc1a-96ab6c5d9c2c None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:52.070569 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-42f5fb66-e98e-41b4-bc1a-96ab6c5d9c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:52.150535 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd55805-ecf0-4542-b09f-07ecf181607b service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:52.156656 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1395/2789] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:16:52.168863 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5fb66-e98e-41b4-bc1a-96ab6c5d9c2c None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:52.171112 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-42f5fb66-e98e-41b4-bc1a-96ab6c5d9c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:52.199480 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:52.206196 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:52.240628 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80f247-b00c-4d8e-99a6-8eea7fee845d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:52.241516 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80f247-b00c-4d8e-99a6-8eea7fee845d service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:52.241906 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80f247-b00c-4d8e-99a6-8eea7fee845d service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:52.242218 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80f247-b00c-4d8e-99a6-8eea7fee845d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:52.245360 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4f80f247-b00c-4d8e-99a6-8eea7fee845d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:52.255820 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5fb66-e98e-41b4-bc1a-96ab6c5d9c2c None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:52.259673 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80f247-b00c-4d8e-99a6-8eea7fee845d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:52.261378 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80f247-b00c-4d8e-99a6-8eea7fee845d service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:52.263439 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ba53d-1407-46af-9468-4d8dd527c453 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:52.264101 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ba53d-1407-46af-9468-4d8dd527c453 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:52.264387 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ba53d-1407-46af-9468-4d8dd527c453 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:52.265234 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ba53d-1407-46af-9468-4d8dd527c453 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:52.273669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1395/2790] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:52.277046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ba53d-1407-46af-9468-4d8dd527c453 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:52.278495 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ba53d-1407-46af-9468-4d8dd527c453 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:52.289385 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1396/2791] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:52.992581 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80f247-b00c-4d8e-99a6-8eea7fee845d service nova] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:52.994337 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4f80f247-b00c-4d8e-99a6-8eea7fee845d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:53.010057 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80f247-b00c-4d8e-99a6-8eea7fee845d service nova] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:53.015393 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6838e-d04f-4666-b08d-77e5219d3ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:53.016079 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6838e-d04f-4666-b08d-77e5219d3ea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:53.016358 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6838e-d04f-4666-b08d-77e5219d3ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:53.016695 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6838e-d04f-4666-b08d-77e5219d3ea2 admin admin] PATH_INFO: `/v3/projects/40a7a0aa6c7845ef994d0a4458c85479` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:53.017904 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6838e-d04f-4666-b08d-77e5219d3ea2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40a7a0aa6c7845ef994d0a4458c85479)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:53.020384 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6838e-d04f-4666-b08d-77e5219d3ea2 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:53.072367 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a8a6838e-d04f-4666-b08d-77e5219d3ea2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40a7a0aa6c7845ef994d0a4458c85479', 'request_id': 'req-a8a6838e-d04f-4666-b08d-77e5219d3ea2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:53.081151 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a8a6838e-d04f-4666-b08d-77e5219d3ea2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40a7a0aa6c7845ef994d0a4458c85479', 'request_id': 'req-a8a6838e-d04f-4666-b08d-77e5219d3ea2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:53.087618 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a8a6838e-d04f-4666-b08d-77e5219d3ea2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40a7a0aa6c7845ef994d0a4458c85479', 'request_id': 'req-a8a6838e-d04f-4666-b08d-77e5219d3ea2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:53.101016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a8a6838e-d04f-4666-b08d-77e5219d3ea2 admin admin] The token cache is being invalidate because project 40a7a0aa6c7845ef994d0a4458c85479 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:53.101341 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a8a6838e-d04f-4666-b08d-77e5219d3ea2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:53.101966 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a8a6838e-d04f-4666-b08d-77e5219d3ea2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:53.103466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1396/2792] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:52 2022] DELETE /identity/v3/projects/40a7a0aa6c7845ef994d0a4458c85479 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:53.107582 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ba53d-1407-46af-9468-4d8dd527c453 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:53.117526 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ba53d-1407-46af-9468-4d8dd527c453 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:53.187982 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137c5e9-1d93-41fa-8076-33ed1a08fd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:53.188697 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137c5e9-1d93-41fa-8076-33ed1a08fd94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:53.188912 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137c5e9-1d93-41fa-8076-33ed1a08fd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:53.189190 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137c5e9-1d93-41fa-8076-33ed1a08fd94 admin admin] PATH_INFO: `/v3/users/d5678b18cca4483bbb2f89f767386e08` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:53.190425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137c5e9-1d93-41fa-8076-33ed1a08fd94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5678b18cca4483bbb2f89f767386e08)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:53.192680 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137c5e9-1d93-41fa-8076-33ed1a08fd94 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:53.260474 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2137c5e9-1d93-41fa-8076-33ed1a08fd94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5678b18cca4483bbb2f89f767386e08'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:53.261475 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2137c5e9-1d93-41fa-8076-33ed1a08fd94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5678b18cca4483bbb2f89f767386e08'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:53.272059 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2137c5e9-1d93-41fa-8076-33ed1a08fd94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5678b18cca4483bbb2f89f767386e08'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:53.272762 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2137c5e9-1d93-41fa-8076-33ed1a08fd94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5678b18cca4483bbb2f89f767386e08'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:53.282772 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2137c5e9-1d93-41fa-8076-33ed1a08fd94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5678b18cca4483bbb2f89f767386e08'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:53.304815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1397/2793] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:16:53 2022] DELETE /identity/v3/users/d5678b18cca4483bbb2f89f767386e08 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:53.629450 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6838e-d04f-4666-b08d-77e5219d3ea2 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:53.631043 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a8a6838e-d04f-4666-b08d-77e5219d3ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:53.644949 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6838e-d04f-4666-b08d-77e5219d3ea2 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:53.720398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627ca90-039d-4ebb-aa35-52a96dada427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:53.721125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627ca90-039d-4ebb-aa35-52a96dada427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:53.721650 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627ca90-039d-4ebb-aa35-52a96dada427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:53.721650 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627ca90-039d-4ebb-aa35-52a96dada427 admin admin] PATH_INFO: `/v3/projects/06bc322a172c4f1ea8d5f05ee1667db5` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:53.722903 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f627ca90-039d-4ebb-aa35-52a96dada427 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06bc322a172c4f1ea8d5f05ee1667db5)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:53.724833 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f627ca90-039d-4ebb-aa35-52a96dada427 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:53.784856 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f627ca90-039d-4ebb-aa35-52a96dada427 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06bc322a172c4f1ea8d5f05ee1667db5', 'request_id': 'req-f627ca90-039d-4ebb-aa35-52a96dada427'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:53.790904 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:53.800167 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f627ca90-039d-4ebb-aa35-52a96dada427 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06bc322a172c4f1ea8d5f05ee1667db5', 'request_id': 'req-f627ca90-039d-4ebb-aa35-52a96dada427'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:53.807901 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f627ca90-039d-4ebb-aa35-52a96dada427 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06bc322a172c4f1ea8d5f05ee1667db5', 'request_id': 'req-f627ca90-039d-4ebb-aa35-52a96dada427'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:53.821061 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f627ca90-039d-4ebb-aa35-52a96dada427 admin admin] The token cache is being invalidate because project 06bc322a172c4f1ea8d5f05ee1667db5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:16:53.821355 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f627ca90-039d-4ebb-aa35-52a96dada427 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:53.822011 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-f627ca90-039d-4ebb-aa35-52a96dada427 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:16:53.823498 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1397/2794] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:16:53 2022] DELETE /identity/v3/projects/06bc322a172c4f1ea8d5f05ee1667db5 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:53.828837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f78b2-df2b-4f57-aae5-95736304487f None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:53.829122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f78b2-df2b-4f57-aae5-95736304487f None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:53.829390 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f78b2-df2b-4f57-aae5-95736304487f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:53.848806 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-747f78b2-df2b-4f57-aae5-95736304487f None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:53.850015 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-747f78b2-df2b-4f57-aae5-95736304487f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:53.883192 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:53.912830 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1398/2795] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:53.920680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07c6fe-02c5-4630-8b97-017a292f78bf None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:53.921519 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07c6fe-02c5-4630-8b97-017a292f78bf None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:53.921801 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07c6fe-02c5-4630-8b97-017a292f78bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:53.941646 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-4e07c6fe-02c5-4630-8b97-017a292f78bf None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:53.942874 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4e07c6fe-02c5-4630-8b97-017a292f78bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:53.951001 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1398/2796] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:53.955383 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f78b2-df2b-4f57-aae5-95736304487f None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:53.965295 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f78b2-df2b-4f57-aae5-95736304487f None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:53.969927 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48500879-c200-439b-abca-ab46d9091605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:53.970536 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48500879-c200-439b-abca-ab46d9091605 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:53.970815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48500879-c200-439b-abca-ab46d9091605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:53.971086 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48500879-c200-439b-abca-ab46d9091605 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:53.971760 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48500879-c200-439b-abca-ab46d9091605 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:16:53.972069 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48500879-c200-439b-abca-ab46d9091605 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:53.973666 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48500879-c200-439b-abca-ab46d9091605 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:53.986608 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1399/2797] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:16:53.990957 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e07c6fe-02c5-4630-8b97-017a292f78bf None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:54.001015 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e07c6fe-02c5-4630-8b97-017a292f78bf None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:54.005935 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c144a82-6e81-425e-be69-7c7df96d7b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:54.006544 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c144a82-6e81-425e-be69-7c7df96d7b94 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:54.006854 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c144a82-6e81-425e-be69-7c7df96d7b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:54.007127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c144a82-6e81-425e-be69-7c7df96d7b94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:54.007820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c144a82-6e81-425e-be69-7c7df96d7b94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:54.010000 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c144a82-6e81-425e-be69-7c7df96d7b94 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:54.038219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1399/2798] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:53 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:54.042189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48500879-c200-439b-abca-ab46d9091605 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:54.053086 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48500879-c200-439b-abca-ab46d9091605 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:54.112469 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901c776-5eef-4675-8f64-9fed409c10f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:54.113078 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901c776-5eef-4675-8f64-9fed409c10f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:54.113358 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901c776-5eef-4675-8f64-9fed409c10f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:54.113633 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901c776-5eef-4675-8f64-9fed409c10f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:54.114277 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901c776-5eef-4675-8f64-9fed409c10f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:54.116237 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901c776-5eef-4675-8f64-9fed409c10f2 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:54.138880 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1400/2799] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:54 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:16:54.143641 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c144a82-6e81-425e-be69-7c7df96d7b94 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:54.155948 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c144a82-6e81-425e-be69-7c7df96d7b94 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:54.160624 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88475a-1d51-4dce-9f6c-1986ec8e7f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:54.161227 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88475a-1d51-4dce-9f6c-1986ec8e7f82 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:54.161511 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88475a-1d51-4dce-9f6c-1986ec8e7f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:54.161790 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88475a-1d51-4dce-9f6c-1986ec8e7f82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:54.162431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88475a-1d51-4dce-9f6c-1986ec8e7f82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:54.164078 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88475a-1d51-4dce-9f6c-1986ec8e7f82 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:54.175017 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1400/2800] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:54.179031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901c776-5eef-4675-8f64-9fed409c10f2 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:54.189669 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901c776-5eef-4675-8f64-9fed409c10f2 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:54.195013 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e4318e-e48e-48ea-a160-b25176879c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:54.195721 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4318e-e48e-48ea-a160-b25176879c86 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:54.195998 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4318e-e48e-48ea-a160-b25176879c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:54.196270 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4318e-e48e-48ea-a160-b25176879c86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:54.196924 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e4318e-e48e-48ea-a160-b25176879c86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:54.198549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e4318e-e48e-48ea-a160-b25176879c86 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:54.210573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1401/2801] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:16:54.214985 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88475a-1d51-4dce-9f6c-1986ec8e7f82 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:54.226251 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88475a-1d51-4dce-9f6c-1986ec8e7f82 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:54.231075 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff0aea-91f4-409d-b042-f9889b1f1c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:54.231790 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff0aea-91f4-409d-b042-f9889b1f1c0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:54.232077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff0aea-91f4-409d-b042-f9889b1f1c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:54.232356 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff0aea-91f4-409d-b042-f9889b1f1c0a admin admin] PATH_INFO: `/v3/projects/fb8c16a675874a4e84d12481c79047c4/users/8547cc53150841299e1b74565701739d/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:54.250609 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff0aea-91f4-409d-b042-f9889b1f1c0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb8c16a675874a4e84d12481c79047c4, user_id=8547cc53150841299e1b74565701739d, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:54.253327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff0aea-91f4-409d-b042-f9889b1f1c0a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:54.267704 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1401/2802] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:54 2022] PUT /identity/v3/projects/fb8c16a675874a4e84d12481c79047c4/users/8547cc53150841299e1b74565701739d/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:54.272737 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5d3d0-9596-46eb-8dbd-18ab9899a1f0 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:54.273024 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5d3d0-9596-46eb-8dbd-18ab9899a1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:54.273326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5d3d0-9596-46eb-8dbd-18ab9899a1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:54.293301 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-79e5d3d0-9596-46eb-8dbd-18ab9899a1f0 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:54.294522 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-79e5d3d0-9596-46eb-8dbd-18ab9899a1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:54.361627 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1402/2803] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:54.371164 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff0aea-91f4-409d-b042-f9889b1f1c0a admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:54.372881 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-dcff0aea-91f4-409d-b042-f9889b1f1c0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:54.438819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff0aea-91f4-409d-b042-f9889b1f1c0a admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:54.442770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debf75a-30b8-46c1-b48d-15512375093e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:54.443387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debf75a-30b8-46c1-b48d-15512375093e service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:54.443697 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debf75a-30b8-46c1-b48d-15512375093e service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:54.443973 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debf75a-30b8-46c1-b48d-15512375093e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:54.454007 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debf75a-30b8-46c1-b48d-15512375093e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:54.455435 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debf75a-30b8-46c1-b48d-15512375093e service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:54.465037 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1402/2804] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:55.093594 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3353eab-126c-45ba-8b17-34c9760182f7 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:55.093879 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3353eab-126c-45ba-8b17-34c9760182f7 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:55.094182 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3353eab-126c-45ba-8b17-34c9760182f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:55.124189 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e3353eab-126c-45ba-8b17-34c9760182f7 None None] MFA Rules not processed for user `8547cc53150841299e1b74565701739d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:55.125769 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e3353eab-126c-45ba-8b17-34c9760182f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:55.160934 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:55.195420 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1403/2805] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:55.199793 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debf75a-30b8-46c1-b48d-15512375093e service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:55.209830 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debf75a-30b8-46c1-b48d-15512375093e service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:55.214389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9c7de-debe-4f15-b787-80919ccc2e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:55.215049 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9c7de-debe-4f15-b787-80919ccc2e48 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:55.215335 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9c7de-debe-4f15-b787-80919ccc2e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:55.215625 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9c7de-debe-4f15-b787-80919ccc2e48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:55.216266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f9c7de-debe-4f15-b787-80919ccc2e48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:55.218833 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f9c7de-debe-4f15-b787-80919ccc2e48 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:55.247192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1403/2806] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:16:55 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:55.251364 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3353eab-126c-45ba-8b17-34c9760182f7 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:55.262476 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3353eab-126c-45ba-8b17-34c9760182f7 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:55.320438 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba64a1f-a038-4a65-b4e8-973f54e46126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:55.321245 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba64a1f-a038-4a65-b4e8-973f54e46126 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:55.321594 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba64a1f-a038-4a65-b4e8-973f54e46126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:55.321918 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba64a1f-a038-4a65-b4e8-973f54e46126 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:55.323595 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba64a1f-a038-4a65-b4e8-973f54e46126 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:55.324794 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba64a1f-a038-4a65-b4e8-973f54e46126 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:55.348560 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1404/2807] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:16:55 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:55.352682 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9c7de-debe-4f15-b787-80919ccc2e48 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:55.363824 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9c7de-debe-4f15-b787-80919ccc2e48 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:55.368571 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2b9a5-96dd-4282-b4f5-3592a866d166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:55.369160 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2b9a5-96dd-4282-b4f5-3592a866d166 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:55.369445 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2b9a5-96dd-4282-b4f5-3592a866d166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:55.369718 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2b9a5-96dd-4282-b4f5-3592a866d166 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:55.370360 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2b9a5-96dd-4282-b4f5-3592a866d166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:55.372002 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2b9a5-96dd-4282-b4f5-3592a866d166 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:55.385192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1404/2808] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:55.390271 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba64a1f-a038-4a65-b4e8-973f54e46126 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:55.401715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba64a1f-a038-4a65-b4e8-973f54e46126 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:55.407704 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464da095-dc3a-490b-b9f3-a09a2348a2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:55.409359 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464da095-dc3a-490b-b9f3-a09a2348a2f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:55.409653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464da095-dc3a-490b-b9f3-a09a2348a2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:55.409956 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464da095-dc3a-490b-b9f3-a09a2348a2f7 admin admin] PATH_INFO: `/v3/projects/4b50701d1d15448385ea7c4ebbe7cea6/users/9d560321687d49eba0c4e7abedc59ea9/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:55.428574 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464da095-dc3a-490b-b9f3-a09a2348a2f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b50701d1d15448385ea7c4ebbe7cea6, user_id=9d560321687d49eba0c4e7abedc59ea9, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:55.432884 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464da095-dc3a-490b-b9f3-a09a2348a2f7 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:55.453820 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1405/2809] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:55 2022] PUT /identity/v3/projects/4b50701d1d15448385ea7c4ebbe7cea6/users/9d560321687d49eba0c4e7abedc59ea9/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:16:55.457808 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2b9a5-96dd-4282-b4f5-3592a866d166 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:55.478063 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2b9a5-96dd-4282-b4f5-3592a866d166 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:55.532318 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:55.552163 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8dcd30-d6cb-40b7-a3c6-67d45ca3bfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:55.552888 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8dcd30-d6cb-40b7-a3c6-67d45ca3bfcf admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:55.553238 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8dcd30-d6cb-40b7-a3c6-67d45ca3bfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:55.553521 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8dcd30-d6cb-40b7-a3c6-67d45ca3bfcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:55.554244 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8dcd30-d6cb-40b7-a3c6-67d45ca3bfcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:55.555954 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8dcd30-d6cb-40b7-a3c6-67d45ca3bfcf admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:55.567613 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1405/2810] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:16:55.570513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464da095-dc3a-490b-b9f3-a09a2348a2f7 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:55.581518 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464da095-dc3a-490b-b9f3-a09a2348a2f7 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:55.586222 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f026fd4-877a-4d66-b4d1-05c4d482d167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:55.586880 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f026fd4-877a-4d66-b4d1-05c4d482d167 admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:55.587162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f026fd4-877a-4d66-b4d1-05c4d482d167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:55.587438 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f026fd4-877a-4d66-b4d1-05c4d482d167 admin admin] PATH_INFO: `/v3/projects/4b50701d1d15448385ea7c4ebbe7cea6/users/9d560321687d49eba0c4e7abedc59ea9/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:55.589478 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f026fd4-877a-4d66-b4d1-05c4d482d167 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b50701d1d15448385ea7c4ebbe7cea6, user_id=9d560321687d49eba0c4e7abedc59ea9, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:55.592165 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f026fd4-877a-4d66-b4d1-05c4d482d167 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:55.603813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1406/2811] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:16:55 2022] PUT /identity/v3/projects/4b50701d1d15448385ea7c4ebbe7cea6/users/9d560321687d49eba0c4e7abedc59ea9/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:16:56.247471 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191413aa-80b0-4702-ad54-dc25eb1a3a2b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:56.248021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191413aa-80b0-4702-ad54-dc25eb1a3a2b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:56.248021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191413aa-80b0-4702-ad54-dc25eb1a3a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:56.287232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-191413aa-80b0-4702-ad54-dc25eb1a3a2b None None] MFA Rules not processed for user `9d560321687d49eba0c4e7abedc59ea9`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:56.288429 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-191413aa-80b0-4702-ad54-dc25eb1a3a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:56.367818 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1406/2812] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:16:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:16:56.379986 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f026fd4-877a-4d66-b4d1-05c4d482d167 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:56.381723 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2f026fd4-877a-4d66-b4d1-05c4d482d167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:56.470588 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f026fd4-877a-4d66-b4d1-05c4d482d167 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:56.474706 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9053e-07de-4bd5-8424-44b8a02e9e2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:56.475326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9053e-07de-4bd5-8424-44b8a02e9e2b service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:56.475639 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9053e-07de-4bd5-8424-44b8a02e9e2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:56.475914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9053e-07de-4bd5-8424-44b8a02e9e2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:56.522811 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:56.541529 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b9053e-07de-4bd5-8424-44b8a02e9e2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:56.542980 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b9053e-07de-4bd5-8424-44b8a02e9e2b service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:56.553350 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1407/2813] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:16:57.958844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191413aa-80b0-4702-ad54-dc25eb1a3a2b None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:57.969012 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191413aa-80b0-4702-ad54-dc25eb1a3a2b None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:57.972514 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26848ef-5539-41e1-981a-5376bff85036 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:57.973109 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26848ef-5539-41e1-981a-5376bff85036 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:57.973387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26848ef-5539-41e1-981a-5376bff85036 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:57.973659 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26848ef-5539-41e1-981a-5376bff85036 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:57.982255 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26848ef-5539-41e1-981a-5376bff85036 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:57.984284 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26848ef-5539-41e1-981a-5376bff85036 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:57.993876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1407/2814] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:16:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:17:03.184226 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf89e41-6e6e-447e-9166-8e5b79a83c04 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:03.184711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf89e41-6e6e-447e-9166-8e5b79a83c04 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:03.184812 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf89e41-6e6e-447e-9166-8e5b79a83c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:03.240492 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7cf89e41-6e6e-447e-9166-8e5b79a83c04 None None] MFA Rules not processed for user `ceccc909529042539dd2d5015058d8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:03.242191 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7cf89e41-6e6e-447e-9166-8e5b79a83c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:03.331431 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1408/2815] 10.222.0.42 () {58 vars in 967 bytes} [Wed Sep 14 01:17:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:17:03.389180 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26848ef-5539-41e1-981a-5376bff85036 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:03.390556 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d26848ef-5539-41e1-981a-5376bff85036 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:03.498846 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26848ef-5539-41e1-981a-5376bff85036 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:03.504840 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cdbacd-c9bb-4d86-9054-b7c7688a6f4e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e52776d9b4894af9b85a24136a8255e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:03.505448 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdbacd-c9bb-4d86-9054-b7c7688a6f4e service swift] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:03.505730 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdbacd-c9bb-4d86-9054-b7c7688a6f4e service swift] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:03.506005 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdbacd-c9bb-4d86-9054-b7c7688a6f4e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:03.514365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cdbacd-c9bb-4d86-9054-b7c7688a6f4e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:03.516221 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cdbacd-c9bb-4d86-9054-b7c7688a6f4e service swift] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:03.524242 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1408/2816] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 01:17:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 01:17:10.588831 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf89e41-6e6e-447e-9166-8e5b79a83c04 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:10.590317 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7cf89e41-6e6e-447e-9166-8e5b79a83c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:10.681123 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf89e41-6e6e-447e-9166-8e5b79a83c04 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:10.686043 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06132470-98a5-4eea-b66c-b5d746cc99e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:10.686878 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06132470-98a5-4eea-b66c-b5d746cc99e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:10.687281 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06132470-98a5-4eea-b66c-b5d746cc99e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:10.687683 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06132470-98a5-4eea-b66c-b5d746cc99e1 admin admin] PATH_INFO: `/v3/users/f8d750f711d34f80816f124cb3d1f4b4` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:10.689283 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06132470-98a5-4eea-b66c-b5d746cc99e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8d750f711d34f80816f124cb3d1f4b4)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:10.691311 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06132470-98a5-4eea-b66c-b5d746cc99e1 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:10.771581 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-06132470-98a5-4eea-b66c-b5d746cc99e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8d750f711d34f80816f124cb3d1f4b4'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:10.771581 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-06132470-98a5-4eea-b66c-b5d746cc99e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8d750f711d34f80816f124cb3d1f4b4'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:10.779241 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-06132470-98a5-4eea-b66c-b5d746cc99e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8d750f711d34f80816f124cb3d1f4b4'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:10.779982 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-06132470-98a5-4eea-b66c-b5d746cc99e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8d750f711d34f80816f124cb3d1f4b4'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:10.782503 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:10.793549 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-06132470-98a5-4eea-b66c-b5d746cc99e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8d750f711d34f80816f124cb3d1f4b4'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:10.830450 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1409/2817] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:17:10 2022] DELETE /identity/v3/users/f8d750f711d34f80816f124cb3d1f4b4 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:11.031492 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cdbacd-c9bb-4d86-9054-b7c7688a6f4e service swift] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:11.053234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cdbacd-c9bb-4d86-9054-b7c7688a6f4e service swift] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:11.084280 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:11.130299 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72768c8-8f4a-4826-8430-e4580e7e42af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:11.130869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72768c8-8f4a-4826-8430-e4580e7e42af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:11.131176 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72768c8-8f4a-4826-8430-e4580e7e42af admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:11.131450 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72768c8-8f4a-4826-8430-e4580e7e42af admin admin] PATH_INFO: `/v3/projects/f93905695d7a4f8495534c38a7bebfa4` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:11.132611 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72768c8-8f4a-4826-8430-e4580e7e42af admin admin] RBAC: Authorizing `identity:delete_project(project_id=f93905695d7a4f8495534c38a7bebfa4)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:11.134787 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72768c8-8f4a-4826-8430-e4580e7e42af admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:11.185834 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c72768c8-8f4a-4826-8430-e4580e7e42af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f93905695d7a4f8495534c38a7bebfa4', 'request_id': 'req-c72768c8-8f4a-4826-8430-e4580e7e42af'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:11.194578 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c72768c8-8f4a-4826-8430-e4580e7e42af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f93905695d7a4f8495534c38a7bebfa4', 'request_id': 'req-c72768c8-8f4a-4826-8430-e4580e7e42af'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:11.200791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c72768c8-8f4a-4826-8430-e4580e7e42af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f93905695d7a4f8495534c38a7bebfa4', 'request_id': 'req-c72768c8-8f4a-4826-8430-e4580e7e42af'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:11.215833 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c72768c8-8f4a-4826-8430-e4580e7e42af admin admin] The token cache is being invalidate because project f93905695d7a4f8495534c38a7bebfa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:17:11.216124 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c72768c8-8f4a-4826-8430-e4580e7e42af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:11.216819 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-c72768c8-8f4a-4826-8430-e4580e7e42af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:11.218313 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1409/2818] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:17:11 2022] DELETE /identity/v3/projects/f93905695d7a4f8495534c38a7bebfa4 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:11.222539 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06132470-98a5-4eea-b66c-b5d746cc99e1 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:11.224003 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-06132470-98a5-4eea-b66c-b5d746cc99e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:11.239668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06132470-98a5-4eea-b66c-b5d746cc99e1 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:11.317405 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea6476-885d-4282-9b55-5e302653d6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:11.318054 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea6476-885d-4282-9b55-5e302653d6c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:11.318333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea6476-885d-4282-9b55-5e302653d6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:11.318605 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea6476-885d-4282-9b55-5e302653d6c4 admin admin] PATH_INFO: `/v3/users/fb0bb8980df24136942f13f8904b3e2a` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:11.320894 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea6476-885d-4282-9b55-5e302653d6c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb0bb8980df24136942f13f8904b3e2a)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:11.323055 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea6476-885d-4282-9b55-5e302653d6c4 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:11.382124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b5ea6476-885d-4282-9b55-5e302653d6c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb0bb8980df24136942f13f8904b3e2a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:11.383273 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b5ea6476-885d-4282-9b55-5e302653d6c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb0bb8980df24136942f13f8904b3e2a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:11.392326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b5ea6476-885d-4282-9b55-5e302653d6c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb0bb8980df24136942f13f8904b3e2a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:11.397823 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b5ea6476-885d-4282-9b55-5e302653d6c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb0bb8980df24136942f13f8904b3e2a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:11.407426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-b5ea6476-885d-4282-9b55-5e302653d6c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb0bb8980df24136942f13f8904b3e2a'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:11.431276 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1410/2819] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:17:11 2022] DELETE /identity/v3/users/fb0bb8980df24136942f13f8904b3e2a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:17:11.625620 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72768c8-8f4a-4826-8430-e4580e7e42af admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:11.627171 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-c72768c8-8f4a-4826-8430-e4580e7e42af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:11.642279 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72768c8-8f4a-4826-8430-e4580e7e42af admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:11.643169 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea6476-885d-4282-9b55-5e302653d6c4 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:11.644845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b5ea6476-885d-4282-9b55-5e302653d6c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:11.707997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b88886-dde9-49f5-926d-4202147907b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:11.709080 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b88886-dde9-49f5-926d-4202147907b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:11.709564 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b88886-dde9-49f5-926d-4202147907b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:11.709846 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b88886-dde9-49f5-926d-4202147907b0 admin admin] PATH_INFO: `/v3/users/8547cc53150841299e1b74565701739d` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:11.711123 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b88886-dde9-49f5-926d-4202147907b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8547cc53150841299e1b74565701739d)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:11.713824 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b88886-dde9-49f5-926d-4202147907b0 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:11.716044 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:11.732136 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea6476-885d-4282-9b55-5e302653d6c4 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:11.739970 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d437d952-6373-4841-a89d-27085da89d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:11.740703 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437d952-6373-4841-a89d-27085da89d15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:11.741008 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437d952-6373-4841-a89d-27085da89d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:11.741286 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437d952-6373-4841-a89d-27085da89d15 admin admin] PATH_INFO: `/v3/projects/cae52d32e65b4630be939de99ecc9d0f` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:11.742584 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d437d952-6373-4841-a89d-27085da89d15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cae52d32e65b4630be939de99ecc9d0f)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:11.744898 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d437d952-6373-4841-a89d-27085da89d15 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:11.765240 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d7b88886-dde9-49f5-926d-4202147907b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8547cc53150841299e1b74565701739d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:11.766520 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d7b88886-dde9-49f5-926d-4202147907b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8547cc53150841299e1b74565701739d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:11.774828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d7b88886-dde9-49f5-926d-4202147907b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8547cc53150841299e1b74565701739d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:11.780233 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:11.784498 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d7b88886-dde9-49f5-926d-4202147907b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8547cc53150841299e1b74565701739d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:11.785977 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d7b88886-dde9-49f5-926d-4202147907b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8547cc53150841299e1b74565701739d'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:11.798228 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d437d952-6373-4841-a89d-27085da89d15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cae52d32e65b4630be939de99ecc9d0f', 'request_id': 'req-d437d952-6373-4841-a89d-27085da89d15'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:11.807190 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d437d952-6373-4841-a89d-27085da89d15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cae52d32e65b4630be939de99ecc9d0f', 'request_id': 'req-d437d952-6373-4841-a89d-27085da89d15'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:11.809400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1410/2820] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:17:11 2022] DELETE /identity/v3/users/8547cc53150841299e1b74565701739d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:11.820255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d437d952-6373-4841-a89d-27085da89d15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cae52d32e65b4630be939de99ecc9d0f', 'request_id': 'req-d437d952-6373-4841-a89d-27085da89d15'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:11.838144 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d437d952-6373-4841-a89d-27085da89d15 admin admin] The token cache is being invalidate because project cae52d32e65b4630be939de99ecc9d0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:17:11.838432 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d437d952-6373-4841-a89d-27085da89d15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:11.839102 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-d437d952-6373-4841-a89d-27085da89d15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:11.840949 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1411/2821] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:17:11 2022] DELETE /identity/v3/projects/cae52d32e65b4630be939de99ecc9d0f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:11.845986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6aa8c3-e160-4d55-b45b-2940557f7a3d None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:11.846280 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6aa8c3-e160-4d55-b45b-2940557f7a3d None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:11.846562 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6aa8c3-e160-4d55-b45b-2940557f7a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:11.866828 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-7f6aa8c3-e160-4d55-b45b-2940557f7a3d None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:11.868146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7f6aa8c3-e160-4d55-b45b-2940557f7a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:11.933016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1411/2822] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:17:11.941361 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccd538-ed62-48c6-81bc-f312215e3bdc None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:11.941702 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccd538-ed62-48c6-81bc-f312215e3bdc None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:11.941957 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccd538-ed62-48c6-81bc-f312215e3bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:11.963260 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-e2ccd538-ed62-48c6-81bc-f312215e3bdc None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:11.964524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e2ccd538-ed62-48c6-81bc-f312215e3bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:11.973607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1412/2823] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:17:11.978369 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6aa8c3-e160-4d55-b45b-2940557f7a3d None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:11.989290 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6aa8c3-e160-4d55-b45b-2940557f7a3d None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:11.991054 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ccd538-ed62-48c6-81bc-f312215e3bdc None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:11.992496 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-e2ccd538-ed62-48c6-81bc-f312215e3bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:11.994418 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755cf50f-41e5-4fa7-8437-11c50cbb41c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:11.995122 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755cf50f-41e5-4fa7-8437-11c50cbb41c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:11.995525 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755cf50f-41e5-4fa7-8437-11c50cbb41c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:11.996602 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755cf50f-41e5-4fa7-8437-11c50cbb41c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:11.997545 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755cf50f-41e5-4fa7-8437-11c50cbb41c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:11.997947 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755cf50f-41e5-4fa7-8437-11c50cbb41c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:12.000975 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755cf50f-41e5-4fa7-8437-11c50cbb41c4 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:12.011033 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ccd538-ed62-48c6-81bc-f312215e3bdc None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:12.016923 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1412/2824] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:12.020801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd104ce-bd28-4f07-bb14-d647c89f8279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:12.021187 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755cf50f-41e5-4fa7-8437-11c50cbb41c4 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:12.021693 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd104ce-bd28-4f07-bb14-d647c89f8279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:12.022093 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd104ce-bd28-4f07-bb14-d647c89f8279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:12.022479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd104ce-bd28-4f07-bb14-d647c89f8279 admin admin] PATH_INFO: `/v3/projects/fb8c16a675874a4e84d12481c79047c4` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:12.023933 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd104ce-bd28-4f07-bb14-d647c89f8279 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb8c16a675874a4e84d12481c79047c4)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:12.026555 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd104ce-bd28-4f07-bb14-d647c89f8279 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:12.031435 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755cf50f-41e5-4fa7-8437-11c50cbb41c4 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:12.038321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5537eb5b-936d-495d-b852-bc4eb7dae507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:12.039197 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537eb5b-936d-495d-b852-bc4eb7dae507 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:12.039480 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537eb5b-936d-495d-b852-bc4eb7dae507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:12.039794 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537eb5b-936d-495d-b852-bc4eb7dae507 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:12.040477 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5537eb5b-936d-495d-b852-bc4eb7dae507 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:12.042377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5537eb5b-936d-495d-b852-bc4eb7dae507 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:12.071443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1413/2825] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:17:12 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:17:12.075803 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5537eb5b-936d-495d-b852-bc4eb7dae507 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:12.089381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dcd104ce-bd28-4f07-bb14-d647c89f8279 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb8c16a675874a4e84d12481c79047c4', 'request_id': 'req-dcd104ce-bd28-4f07-bb14-d647c89f8279'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:12.097739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dcd104ce-bd28-4f07-bb14-d647c89f8279 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb8c16a675874a4e84d12481c79047c4', 'request_id': 'req-dcd104ce-bd28-4f07-bb14-d647c89f8279'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:12.103493 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dcd104ce-bd28-4f07-bb14-d647c89f8279 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb8c16a675874a4e84d12481c79047c4', 'request_id': 'req-dcd104ce-bd28-4f07-bb14-d647c89f8279'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:12.115448 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dcd104ce-bd28-4f07-bb14-d647c89f8279 admin admin] The token cache is being invalidate because project fb8c16a675874a4e84d12481c79047c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:17:12.115778 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dcd104ce-bd28-4f07-bb14-d647c89f8279 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:12.116453 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dcd104ce-bd28-4f07-bb14-d647c89f8279 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:12.118476 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1413/2826] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:17:11 2022] DELETE /identity/v3/projects/fb8c16a675874a4e84d12481c79047c4 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:17:12.122325 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd104ce-bd28-4f07-bb14-d647c89f8279 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:12.123796 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dcd104ce-bd28-4f07-bb14-d647c89f8279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:12.139995 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5537eb5b-936d-495d-b852-bc4eb7dae507 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:12.185096 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:12.185967 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:12.191323 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd104ce-bd28-4f07-bb14-d647c89f8279 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:12.197680 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232639c-c3ec-46de-aa76-79e936958992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:12.199169 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232639c-c3ec-46de-aa76-79e936958992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:12.199479 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232639c-c3ec-46de-aa76-79e936958992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:12.199828 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232639c-c3ec-46de-aa76-79e936958992 admin admin] PATH_INFO: `/v3/users/9d560321687d49eba0c4e7abedc59ea9` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:12.201441 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da20487-2f9d-4aa0-a1b7-03a4db270a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:12.201776 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f232639c-c3ec-46de-aa76-79e936958992 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d560321687d49eba0c4e7abedc59ea9)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:12.202017 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da20487-2f9d-4aa0-a1b7-03a4db270a90 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:12.202298 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da20487-2f9d-4aa0-a1b7-03a4db270a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:12.202563 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da20487-2f9d-4aa0-a1b7-03a4db270a90 admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:12.203188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da20487-2f9d-4aa0-a1b7-03a4db270a90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:12.205302 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f232639c-c3ec-46de-aa76-79e936958992 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:12.205751 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da20487-2f9d-4aa0-a1b7-03a4db270a90 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:12.229907 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1414/2827] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:17:12 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:17:12.234248 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da20487-2f9d-4aa0-a1b7-03a4db270a90 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:12.236236 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4da20487-2f9d-4aa0-a1b7-03a4db270a90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:12.248847 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da20487-2f9d-4aa0-a1b7-03a4db270a90 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:12.255669 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff762119-3d74-4754-a8af-ea3a40abd1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:12.256374 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff762119-3d74-4754-a8af-ea3a40abd1ff admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:12.256660 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff762119-3d74-4754-a8af-ea3a40abd1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:12.256968 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff762119-3d74-4754-a8af-ea3a40abd1ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:12.257794 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff762119-3d74-4754-a8af-ea3a40abd1ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:12.259718 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff762119-3d74-4754-a8af-ea3a40abd1ff admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:12.260726 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f232639c-c3ec-46de-aa76-79e936958992 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d560321687d49eba0c4e7abedc59ea9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:12.261521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f232639c-c3ec-46de-aa76-79e936958992 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d560321687d49eba0c4e7abedc59ea9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:12.269167 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f232639c-c3ec-46de-aa76-79e936958992 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d560321687d49eba0c4e7abedc59ea9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:12.269866 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f232639c-c3ec-46de-aa76-79e936958992 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d560321687d49eba0c4e7abedc59ea9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:12.271016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1415/2828] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:17:12.275171 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff762119-3d74-4754-a8af-ea3a40abd1ff admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:12.277024 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-ff762119-3d74-4754-a8af-ea3a40abd1ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:12.277652 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-f232639c-c3ec-46de-aa76-79e936958992 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d560321687d49eba0c4e7abedc59ea9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:12.290561 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff762119-3d74-4754-a8af-ea3a40abd1ff admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:12.295663 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1414/2829] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:17:12 2022] DELETE /identity/v3/users/9d560321687d49eba0c4e7abedc59ea9 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:17:12.295795 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5f7b1-96c8-4437-91ff-d062816189ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:12.296325 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5f7b1-96c8-4437-91ff-d062816189ae admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:12.296604 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5f7b1-96c8-4437-91ff-d062816189ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:12.296878 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5f7b1-96c8-4437-91ff-d062816189ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:12.297554 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5f7b1-96c8-4437-91ff-d062816189ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:12.299315 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5f7b1-96c8-4437-91ff-d062816189ae admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:12.310125 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1416/2830] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:17:12.315149 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232639c-c3ec-46de-aa76-79e936958992 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:12.379757 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232639c-c3ec-46de-aa76-79e936958992 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:12.384722 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84575af3-2841-496c-877f-7604e5f0ec1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:12.385380 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84575af3-2841-496c-877f-7604e5f0ec1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:12.385657 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84575af3-2841-496c-877f-7604e5f0ec1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:12.385926 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84575af3-2841-496c-877f-7604e5f0ec1f admin admin] PATH_INFO: `/v3/projects/b2d80d1c2758499da66a93e91f9c1311/users/9e0e8bab9899415c91d3b7122ab0a1d0/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:12.401669 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84575af3-2841-496c-877f-7604e5f0ec1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2d80d1c2758499da66a93e91f9c1311, user_id=9e0e8bab9899415c91d3b7122ab0a1d0, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:12.404634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84575af3-2841-496c-877f-7604e5f0ec1f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:12.417785 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1415/2831] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:17:12 2022] PUT /identity/v3/projects/b2d80d1c2758499da66a93e91f9c1311/users/9e0e8bab9899415c91d3b7122ab0a1d0/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:12.423630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55dca0a-0256-4261-a04a-6601fe3069fd None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:12.423921 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55dca0a-0256-4261-a04a-6601fe3069fd None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:12.424195 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55dca0a-0256-4261-a04a-6601fe3069fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:12.443488 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-f55dca0a-0256-4261-a04a-6601fe3069fd None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:12.444655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f55dca0a-0256-4261-a04a-6601fe3069fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:12.477444 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84575af3-2841-496c-877f-7604e5f0ec1f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:12.482876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-84575af3-2841-496c-877f-7604e5f0ec1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:12.524027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1417/2832] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:17:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:17:12.531007 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55dca0a-0256-4261-a04a-6601fe3069fd None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:12.533823 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f55dca0a-0256-4261-a04a-6601fe3069fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:12.561636 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84575af3-2841-496c-877f-7604e5f0ec1f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:12.567418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b14bb-9c07-4f4e-b258-954d56c628a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:12.568121 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b14bb-9c07-4f4e-b258-954d56c628a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:12.568402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b14bb-9c07-4f4e-b258-954d56c628a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:12.568704 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b14bb-9c07-4f4e-b258-954d56c628a0 admin admin] PATH_INFO: `/v3/projects/4b50701d1d15448385ea7c4ebbe7cea6` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:12.569850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b14bb-9c07-4f4e-b258-954d56c628a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b50701d1d15448385ea7c4ebbe7cea6)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:12.571778 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b14bb-9c07-4f4e-b258-954d56c628a0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:12.573317 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:12.593664 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55dca0a-0256-4261-a04a-6601fe3069fd None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:12.598260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d7a58-16b8-4466-b191-0007c7428c7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:12.598899 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d7a58-16b8-4466-b191-0007c7428c7c service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:12.599188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d7a58-16b8-4466-b191-0007c7428c7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:12.599466 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d7a58-16b8-4466-b191-0007c7428c7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:12.610465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d7a58-16b8-4466-b191-0007c7428c7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:12.612016 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d7a58-16b8-4466-b191-0007c7428c7c service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:12.621906 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1418/2833] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:17:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:17:12.635796 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ab0b14bb-9c07-4f4e-b258-954d56c628a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b50701d1d15448385ea7c4ebbe7cea6', 'request_id': 'req-ab0b14bb-9c07-4f4e-b258-954d56c628a0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:12.646555 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ab0b14bb-9c07-4f4e-b258-954d56c628a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b50701d1d15448385ea7c4ebbe7cea6', 'request_id': 'req-ab0b14bb-9c07-4f4e-b258-954d56c628a0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:12.653612 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ab0b14bb-9c07-4f4e-b258-954d56c628a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b50701d1d15448385ea7c4ebbe7cea6', 'request_id': 'req-ab0b14bb-9c07-4f4e-b258-954d56c628a0'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:12.667060 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ab0b14bb-9c07-4f4e-b258-954d56c628a0 admin admin] The token cache is being invalidate because project 4b50701d1d15448385ea7c4ebbe7cea6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:17:12.667213 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ab0b14bb-9c07-4f4e-b258-954d56c628a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:12.667906 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-ab0b14bb-9c07-4f4e-b258-954d56c628a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:12.669421 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1416/2834] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:17:12 2022] DELETE /identity/v3/projects/4b50701d1d15448385ea7c4ebbe7cea6 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:17:12.675096 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d01964-3145-463c-96b0-4ad955b7e5ee None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:12.675417 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d01964-3145-463c-96b0-4ad955b7e5ee None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:12.675728 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d01964-3145-463c-96b0-4ad955b7e5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:12.695526 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a2d01964-3145-463c-96b0-4ad955b7e5ee None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:12.696699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a2d01964-3145-463c-96b0-4ad955b7e5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:12.751714 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:12.762055 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1419/2835] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:17:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:17:12.771025 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430cbeb-9330-4813-a441-c5c9c96bc268 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:12.771921 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430cbeb-9330-4813-a441-c5c9c96bc268 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:12.772275 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430cbeb-9330-4813-a441-c5c9c96bc268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:12.793707 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d430cbeb-9330-4813-a441-c5c9c96bc268 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:12.795122 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d430cbeb-9330-4813-a441-c5c9c96bc268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:12.803835 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1417/2836] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:17:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:17:12.808500 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d01964-3145-463c-96b0-4ad955b7e5ee None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:12.819088 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d01964-3145-463c-96b0-4ad955b7e5ee None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:12.824232 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac532362-08c3-4b20-a3f9-1c05db04f5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:12.824831 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac532362-08c3-4b20-a3f9-1c05db04f5cb admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:12.825111 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac532362-08c3-4b20-a3f9-1c05db04f5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:12.825391 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac532362-08c3-4b20-a3f9-1c05db04f5cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:12.826064 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac532362-08c3-4b20-a3f9-1c05db04f5cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81418) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:12.826369 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac532362-08c3-4b20-a3f9-1c05db04f5cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:12.828007 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac532362-08c3-4b20-a3f9-1c05db04f5cb admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:12.840972 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1420/2837] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:12.846467 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430cbeb-9330-4813-a441-c5c9c96bc268 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:12.858749 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430cbeb-9330-4813-a441-c5c9c96bc268 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:12.866048 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ea6f2-80ef-4b2f-99ec-a6764491cbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:12.866679 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ea6f2-80ef-4b2f-99ec-a6764491cbbf admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:12.867029 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ea6f2-80ef-4b2f-99ec-a6764491cbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:12.867270 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ea6f2-80ef-4b2f-99ec-a6764491cbbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:12.867950 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7ea6f2-80ef-4b2f-99ec-a6764491cbbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:12.869847 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7ea6f2-80ef-4b2f-99ec-a6764491cbbf admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:12.905914 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1418/2838] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:17:12 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:17:12.907741 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac532362-08c3-4b20-a3f9-1c05db04f5cb admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:12.918811 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac532362-08c3-4b20-a3f9-1c05db04f5cb admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:12.983212 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c82f8a8-3401-4796-bb29-68e1aab8778a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:12.983868 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82f8a8-3401-4796-bb29-68e1aab8778a admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:12.984157 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82f8a8-3401-4796-bb29-68e1aab8778a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:12.984434 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82f8a8-3401-4796-bb29-68e1aab8778a admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:12.985266 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c82f8a8-3401-4796-bb29-68e1aab8778a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:12.987246 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c82f8a8-3401-4796-bb29-68e1aab8778a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:13.011798 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1421/2839] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:17:12 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:17:13.015676 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ea6f2-80ef-4b2f-99ec-a6764491cbbf admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:13.027423 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ea6f2-80ef-4b2f-99ec-a6764491cbbf admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:13.032594 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cdea3a-83fc-443e-861f-4dd4cfe35ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:13.033336 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdea3a-83fc-443e-861f-4dd4cfe35ceb admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:13.033593 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdea3a-83fc-443e-861f-4dd4cfe35ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:13.033944 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdea3a-83fc-443e-861f-4dd4cfe35ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:13.034691 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cdea3a-83fc-443e-861f-4dd4cfe35ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:13.036300 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cdea3a-83fc-443e-861f-4dd4cfe35ceb admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:13.049889 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1419/2840] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:17:13.051041 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c82f8a8-3401-4796-bb29-68e1aab8778a admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:13.064882 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c82f8a8-3401-4796-bb29-68e1aab8778a admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:13.070374 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82e562-d079-4af2-8c78-62e962102bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:13.070976 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82e562-d079-4af2-8c78-62e962102bda admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:13.071263 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82e562-d079-4af2-8c78-62e962102bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:13.071538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82e562-d079-4af2-8c78-62e962102bda admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:13.072219 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d82e562-d079-4af2-8c78-62e962102bda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:13.073824 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d82e562-d079-4af2-8c78-62e962102bda admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:13.085621 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1422/2841] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:17:13.089803 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cdea3a-83fc-443e-861f-4dd4cfe35ceb admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:13.099880 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cdea3a-83fc-443e-861f-4dd4cfe35ceb admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:13.104504 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c57a37-6b2c-4454-8d05-803699d46dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:13.105497 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c57a37-6b2c-4454-8d05-803699d46dfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:13.105497 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c57a37-6b2c-4454-8d05-803699d46dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:13.105741 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c57a37-6b2c-4454-8d05-803699d46dfc admin admin] PATH_INFO: `/v3/projects/6c0d5a6539764b53852b5741347734a8/users/b1494511685b47519f4aa0674009fbfc/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:13.121311 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c57a37-6b2c-4454-8d05-803699d46dfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c0d5a6539764b53852b5741347734a8, user_id=b1494511685b47519f4aa0674009fbfc, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:13.124333 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c57a37-6b2c-4454-8d05-803699d46dfc admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:13.139201 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1420/2842] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:17:13 2022] PUT /identity/v3/projects/6c0d5a6539764b53852b5741347734a8/users/b1494511685b47519f4aa0674009fbfc/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:13.145045 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17c06e-dcd9-494d-81fd-f23c8ecbc594 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:13.145338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17c06e-dcd9-494d-81fd-f23c8ecbc594 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:13.145611 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17c06e-dcd9-494d-81fd-f23c8ecbc594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:13.166831 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-cc17c06e-dcd9-494d-81fd-f23c8ecbc594 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:13.168146 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-cc17c06e-dcd9-494d-81fd-f23c8ecbc594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:13.232976 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1423/2843] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:17:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:17:13.242926 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c57a37-6b2c-4454-8d05-803699d46dfc admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:13.245167 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-69c57a37-6b2c-4454-8d05-803699d46dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:13.322366 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c57a37-6b2c-4454-8d05-803699d46dfc admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:13.328886 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9aa88-d38b-4228-87ae-e7dec73c1cfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:13.329521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9aa88-d38b-4228-87ae-e7dec73c1cfb service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:13.330143 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9aa88-d38b-4228-87ae-e7dec73c1cfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:13.330388 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9aa88-d38b-4228-87ae-e7dec73c1cfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:13.345588 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9aa88-d38b-4228-87ae-e7dec73c1cfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:13.347049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f9aa88-d38b-4228-87ae-e7dec73c1cfb service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:13.359765 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1421/2844] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:17:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:17:17.942106 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49175b-65ef-442d-8e6e-28d57564fa2e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:17.942608 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49175b-65ef-442d-8e6e-28d57564fa2e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:17.942726 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49175b-65ef-442d-8e6e-28d57564fa2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:17.982602 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-2f49175b-65ef-442d-8e6e-28d57564fa2e None None] MFA Rules not processed for user `9e0e8bab9899415c91d3b7122ab0a1d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:17.983947 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2f49175b-65ef-442d-8e6e-28d57564fa2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:18.063680 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:18.092227 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1424/2845] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:17:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:17:18.092480 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9aa88-d38b-4228-87ae-e7dec73c1cfb service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:18.095425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-40f9aa88-d38b-4228-87ae-e7dec73c1cfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:18.111496 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f9aa88-d38b-4228-87ae-e7dec73c1cfb service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:18.117320 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d17b17-092f-4966-a77e-4614055c32ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:18.117951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d17b17-092f-4966-a77e-4614055c32ea admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:18.118233 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d17b17-092f-4966-a77e-4614055c32ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:18.118512 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d17b17-092f-4966-a77e-4614055c32ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:18.119159 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d17b17-092f-4966-a77e-4614055c32ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:18.121198 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d17b17-092f-4966-a77e-4614055c32ea admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:18.165329 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1422/2846] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:17:18 2022] POST /identity/v3/projects => generated 371 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:17:18.168757 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49175b-65ef-442d-8e6e-28d57564fa2e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:18.187260 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49175b-65ef-442d-8e6e-28d57564fa2e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:18.264678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607fc5c-ce46-414b-ae44-24e79ec86bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:18.265787 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607fc5c-ce46-414b-ae44-24e79ec86bcc admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:18.266965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607fc5c-ce46-414b-ae44-24e79ec86bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:18.267454 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607fc5c-ce46-414b-ae44-24e79ec86bcc admin admin] PATH_INFO: `/v3/users` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:18.268227 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2607fc5c-ce46-414b-ae44-24e79ec86bcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:18.270699 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2607fc5c-ce46-414b-ae44-24e79ec86bcc admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:18.306175 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1425/2847] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:17:18 2022] POST /identity/v3/users => generated 292 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:17:18.309511 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d17b17-092f-4966-a77e-4614055c32ea admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:18.324162 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d17b17-092f-4966-a77e-4614055c32ea admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:18.349217 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52443c-2d36-4036-a2a0-59e7e53a2700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:18.349920 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52443c-2d36-4036-a2a0-59e7e53a2700 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:18.350206 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52443c-2d36-4036-a2a0-59e7e53a2700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:18.350476 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52443c-2d36-4036-a2a0-59e7e53a2700 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:18.351168 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52443c-2d36-4036-a2a0-59e7e53a2700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:18.354311 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52443c-2d36-4036-a2a0-59e7e53a2700 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:18.356571 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:18.380384 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1423/2848] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:17:18.384702 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607fc5c-ce46-414b-ae44-24e79ec86bcc admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:18.400433 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607fc5c-ce46-414b-ae44-24e79ec86bcc admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:18.406037 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a740146-61b7-4617-a6b5-a96cdb40f6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:18.407218 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a740146-61b7-4617-a6b5-a96cdb40f6c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:18.407926 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a740146-61b7-4617-a6b5-a96cdb40f6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:18.408365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a740146-61b7-4617-a6b5-a96cdb40f6c1 admin admin] PATH_INFO: `/v3/projects/e9da155613f948a5a168179f7edc64dc/users/508e8e3ccb30432d9ac0e696dc9a92c6/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:18.437740 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a740146-61b7-4617-a6b5-a96cdb40f6c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9da155613f948a5a168179f7edc64dc, user_id=508e8e3ccb30432d9ac0e696dc9a92c6, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:18.447933 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a740146-61b7-4617-a6b5-a96cdb40f6c1 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:18.476241 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1426/2849] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:17:18 2022] PUT /identity/v3/projects/e9da155613f948a5a168179f7edc64dc/users/508e8e3ccb30432d9ac0e696dc9a92c6/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:18.478972 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52443c-2d36-4036-a2a0-59e7e53a2700 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:18.494440 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52443c-2d36-4036-a2a0-59e7e53a2700 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:18.581663 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da8049-c965-4d78-ab35-ed845b53006b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:18.582297 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da8049-c965-4d78-ab35-ed845b53006b admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:18.582579 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da8049-c965-4d78-ab35-ed845b53006b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:18.582851 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da8049-c965-4d78-ab35-ed845b53006b admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:18.583593 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78da8049-c965-4d78-ab35-ed845b53006b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:18.585169 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78da8049-c965-4d78-ab35-ed845b53006b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:18.606328 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1424/2850] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:17:18.607578 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a740146-61b7-4617-a6b5-a96cdb40f6c1 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:18.619148 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a740146-61b7-4617-a6b5-a96cdb40f6c1 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:18.624744 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab7347-7d22-49df-bc9d-b4e478ce26be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:18.625733 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab7347-7d22-49df-bc9d-b4e478ce26be admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:18.626304 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab7347-7d22-49df-bc9d-b4e478ce26be admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:18.626689 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab7347-7d22-49df-bc9d-b4e478ce26be admin admin] PATH_INFO: `/v3/projects/e9da155613f948a5a168179f7edc64dc/users/508e8e3ccb30432d9ac0e696dc9a92c6/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:18.629547 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab7347-7d22-49df-bc9d-b4e478ce26be admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9da155613f948a5a168179f7edc64dc, user_id=508e8e3ccb30432d9ac0e696dc9a92c6, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:18.632172 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab7347-7d22-49df-bc9d-b4e478ce26be admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:18.644970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1427/2851] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:17:18 2022] PUT /identity/v3/projects/e9da155613f948a5a168179f7edc64dc/users/508e8e3ccb30432d9ac0e696dc9a92c6/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:19.524851 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c391b-2f24-400c-8a70-97732aea3de9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:19.525337 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c391b-2f24-400c-8a70-97732aea3de9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:19.525450 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c391b-2f24-400c-8a70-97732aea3de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:19.570798 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-3c0c391b-2f24-400c-8a70-97732aea3de9 None None] MFA Rules not processed for user `b1494511685b47519f4aa0674009fbfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:19.572912 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-3c0c391b-2f24-400c-8a70-97732aea3de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:19.684724 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1425/2852] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:17:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:17:19.689127 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab7347-7d22-49df-bc9d-b4e478ce26be admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:19.801872 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab7347-7d22-49df-bc9d-b4e478ce26be admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:19.809425 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3125f-f50e-434c-acca-5c2f526ebfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:19.810627 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3125f-f50e-434c-acca-5c2f526ebfd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:19.811153 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3125f-f50e-434c-acca-5c2f526ebfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:19.811774 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3125f-f50e-434c-acca-5c2f526ebfd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:19.813089 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3125f-f50e-434c-acca-5c2f526ebfd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:19.815724 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3125f-f50e-434c-acca-5c2f526ebfd1 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:19.883387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1428/2853] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:17:19 2022] POST /identity/v3/projects => generated 377 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:17:19.884391 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c391b-2f24-400c-8a70-97732aea3de9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:19.898643 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c391b-2f24-400c-8a70-97732aea3de9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:20.028463 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ba542-9749-47dd-9dfd-65dbceb7916a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:20.029925 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ba542-9749-47dd-9dfd-65dbceb7916a admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:20.030379 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ba542-9749-47dd-9dfd-65dbceb7916a admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:20.030812 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ba542-9749-47dd-9dfd-65dbceb7916a admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:20.032733 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ba542-9749-47dd-9dfd-65dbceb7916a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:20.037974 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ba542-9749-47dd-9dfd-65dbceb7916a admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:20.102298 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1426/2854] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:17:19 2022] POST /identity/v3/users => generated 295 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:17:20.105951 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3125f-f50e-434c-acca-5c2f526ebfd1 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:20.119313 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3125f-f50e-434c-acca-5c2f526ebfd1 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:20.125793 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e4062-191a-46c8-8ec6-81470316eec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:20.127121 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e4062-191a-46c8-8ec6-81470316eec4 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:20.127912 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e4062-191a-46c8-8ec6-81470316eec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:20.128779 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e4062-191a-46c8-8ec6-81470316eec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:20.129879 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e4062-191a-46c8-8ec6-81470316eec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:20.134946 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e4062-191a-46c8-8ec6-81470316eec4 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:20.158374 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1429/2855] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:17:20.159176 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ba542-9749-47dd-9dfd-65dbceb7916a admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:20.184124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ba542-9749-47dd-9dfd-65dbceb7916a admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:20.192154 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb36fd-a9e3-4798-afa6-8e0739a0fe9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:20.193277 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb36fd-a9e3-4798-afa6-8e0739a0fe9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:20.193934 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb36fd-a9e3-4798-afa6-8e0739a0fe9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:20.194410 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb36fd-a9e3-4798-afa6-8e0739a0fe9c admin admin] PATH_INFO: `/v3/projects/849e60970e2342b08ce58582752ef719/users/ec177049ed9c43efa589090bf9d42914/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:20.209028 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:20.220518 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb36fd-a9e3-4798-afa6-8e0739a0fe9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=849e60970e2342b08ce58582752ef719, user_id=ec177049ed9c43efa589090bf9d42914, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:20.228776 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb36fd-a9e3-4798-afa6-8e0739a0fe9c admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:20.256407 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1427/2856] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:17:20 2022] PUT /identity/v3/projects/849e60970e2342b08ce58582752ef719/users/ec177049ed9c43efa589090bf9d42914/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:20.260514 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e4062-191a-46c8-8ec6-81470316eec4 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:20.273974 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e4062-191a-46c8-8ec6-81470316eec4 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:20.298638 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:20.358046 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e10c9-290c-47a0-99ae-c4e65cf68286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:20.359046 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e10c9-290c-47a0-99ae-c4e65cf68286 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:20.359424 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e10c9-290c-47a0-99ae-c4e65cf68286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:20.359861 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e10c9-290c-47a0-99ae-c4e65cf68286 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:20.360620 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e10c9-290c-47a0-99ae-c4e65cf68286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:20.362571 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e10c9-290c-47a0-99ae-c4e65cf68286 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:20.378375 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1430/2857] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:17:20.379588 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb36fd-a9e3-4798-afa6-8e0739a0fe9c admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:20.395226 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb36fd-a9e3-4798-afa6-8e0739a0fe9c admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:20.406631 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee756c-62b7-41c9-90a0-b02557e7212d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:20.408084 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee756c-62b7-41c9-90a0-b02557e7212d admin admin] REQUEST_METHOD: `PUT` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:20.408374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee756c-62b7-41c9-90a0-b02557e7212d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:20.408657 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee756c-62b7-41c9-90a0-b02557e7212d admin admin] PATH_INFO: `/v3/projects/849e60970e2342b08ce58582752ef719/users/ec177049ed9c43efa589090bf9d42914/roles/ada268ea63354a0297070369dfe3c6cc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:20.411085 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee756c-62b7-41c9-90a0-b02557e7212d admin admin] RBAC: Authorizing `identity:create_grant(project_id=849e60970e2342b08ce58582752ef719, user_id=ec177049ed9c43efa589090bf9d42914, role_id=ada268ea63354a0297070369dfe3c6cc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:20.414368 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee756c-62b7-41c9-90a0-b02557e7212d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:20.436290 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1428/2858] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:17:20 2022] PUT /identity/v3/projects/849e60970e2342b08ce58582752ef719/users/ec177049ed9c43efa589090bf9d42914/roles/ada268ea63354a0297070369dfe3c6cc => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:25.654591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284258f-08f3-4cef-913c-7a012859f24e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:25.658826 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284258f-08f3-4cef-913c-7a012859f24e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:25.659384 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284258f-08f3-4cef-913c-7a012859f24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:25.741056 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-d284258f-08f3-4cef-913c-7a012859f24e None None] MFA Rules not processed for user `508e8e3ccb30432d9ac0e696dc9a92c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:25.742402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d284258f-08f3-4cef-913c-7a012859f24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:25.865518 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1431/2859] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:17:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:17:25.885172 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee756c-62b7-41c9-90a0-b02557e7212d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:25.890475 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0dee756c-62b7-41c9-90a0-b02557e7212d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:26.106452 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee756c-62b7-41c9-90a0-b02557e7212d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:26.117341 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b950e626-b517-4ca7-a4fe-eca2cb2ad126 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:26.118332 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950e626-b517-4ca7-a4fe-eca2cb2ad126 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:26.118736 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950e626-b517-4ca7-a4fe-eca2cb2ad126 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:26.119161 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950e626-b517-4ca7-a4fe-eca2cb2ad126 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:26.280489 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b950e626-b517-4ca7-a4fe-eca2cb2ad126 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:26.283950 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b950e626-b517-4ca7-a4fe-eca2cb2ad126 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:26.312389 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1429/2860] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:17:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:17:27.291385 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71def78-c06a-4755-887a-d327e530a24b None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:27.292268 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71def78-c06a-4755-887a-d327e530a24b None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:27.292735 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71def78-c06a-4755-887a-d327e530a24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:27.352952 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a71def78-c06a-4755-887a-d327e530a24b None None] MFA Rules not processed for user `ec177049ed9c43efa589090bf9d42914`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:27.354338 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a71def78-c06a-4755-887a-d327e530a24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:27.495245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1432/2861] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:17:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:17:27.519488 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b950e626-b517-4ca7-a4fe-eca2cb2ad126 service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:27.523895 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-b950e626-b517-4ca7-a4fe-eca2cb2ad126 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:27.560924 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b950e626-b517-4ca7-a4fe-eca2cb2ad126 service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:27.567795 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cc077-5159-47d3-9a9d-7bad00ae531a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:27.568720 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cc077-5159-47d3-9a9d-7bad00ae531a service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:27.569470 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cc077-5159-47d3-9a9d-7bad00ae531a service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:27.569920 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cc077-5159-47d3-9a9d-7bad00ae531a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:27.676031 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8cc077-5159-47d3-9a9d-7bad00ae531a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:27.677930 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8cc077-5159-47d3-9a9d-7bad00ae531a service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:27.692047 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1430/2862] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:17:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:17:27.999970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1638d9d-1c9d-4053-ab44-ae3582b89cdc None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:28.001581 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1638d9d-1c9d-4053-ab44-ae3582b89cdc None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:28.001786 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1638d9d-1c9d-4053-ab44-ae3582b89cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:28.036476 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-b1638d9d-1c9d-4053-ab44-ae3582b89cdc None None] MFA Rules not processed for user `9e0e8bab9899415c91d3b7122ab0a1d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:28.041900 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b1638d9d-1c9d-4053-ab44-ae3582b89cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:28.062336 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1433/2863] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:17:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:17:29.725074 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cc077-5159-47d3-9a9d-7bad00ae531a service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:29.726653 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-aa8cc077-5159-47d3-9a9d-7bad00ae531a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:29.742793 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:29.860021 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cc077-5159-47d3-9a9d-7bad00ae531a service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:29.868565 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29721fcf-5b76-49dd-9e0a-232ab861f95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:29.869238 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29721fcf-5b76-49dd-9e0a-232ab861f95d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:29.869671 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29721fcf-5b76-49dd-9e0a-232ab861f95d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:29.869956 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29721fcf-5b76-49dd-9e0a-232ab861f95d admin admin] PATH_INFO: `/v3/users/1740d3d1f6df488ea071b2d8898e548c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:29.871157 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29721fcf-5b76-49dd-9e0a-232ab861f95d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1740d3d1f6df488ea071b2d8898e548c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:29.873137 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29721fcf-5b76-49dd-9e0a-232ab861f95d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:29.923017 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-29721fcf-5b76-49dd-9e0a-232ab861f95d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1740d3d1f6df488ea071b2d8898e548c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:29.923842 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-29721fcf-5b76-49dd-9e0a-232ab861f95d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1740d3d1f6df488ea071b2d8898e548c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:29.931233 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-29721fcf-5b76-49dd-9e0a-232ab861f95d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1740d3d1f6df488ea071b2d8898e548c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:29.931942 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-29721fcf-5b76-49dd-9e0a-232ab861f95d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1740d3d1f6df488ea071b2d8898e548c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:29.940608 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-29721fcf-5b76-49dd-9e0a-232ab861f95d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1740d3d1f6df488ea071b2d8898e548c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:29.958524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1431/2864] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:17:29 2022] DELETE /identity/v3/users/1740d3d1f6df488ea071b2d8898e548c => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:29.962997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b36646-0972-40aa-9860-ad019e833146 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:29.963496 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b36646-0972-40aa-9860-ad019e833146 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:29.963631 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b36646-0972-40aa-9860-ad019e833146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:29.982550 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-33b36646-0972-40aa-9860-ad019e833146 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:29.984991 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-33b36646-0972-40aa-9860-ad019e833146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:30.003910 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=100 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:30.048468 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1434/2865] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:17:30.058462 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29721fcf-5b76-49dd-9e0a-232ab861f95d admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:30.060024 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-29721fcf-5b76-49dd-9e0a-232ab861f95d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:30.117731 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29721fcf-5b76-49dd-9e0a-232ab861f95d admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:30.121502 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb5de9-c0d1-4159-9ab5-9d746da01adb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:30.122111 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb5de9-c0d1-4159-9ab5-9d746da01adb service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:30.122393 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb5de9-c0d1-4159-9ab5-9d746da01adb service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:30.122671 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb5de9-c0d1-4159-9ab5-9d746da01adb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:30.139879 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb5de9-c0d1-4159-9ab5-9d746da01adb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:30.141623 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb5de9-c0d1-4159-9ab5-9d746da01adb service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:30.152030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1432/2866] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:17:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:17:30.474687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b36646-0972-40aa-9860-ad019e833146 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:30.476527 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-33b36646-0972-40aa-9860-ad019e833146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:30.490688 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b36646-0972-40aa-9860-ad019e833146 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:30.496478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742ba4c-beca-4786-8388-e003a64204e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:30.497128 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742ba4c-beca-4786-8388-e003a64204e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:30.497409 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742ba4c-beca-4786-8388-e003a64204e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:30.497687 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742ba4c-beca-4786-8388-e003a64204e5 admin admin] PATH_INFO: `/v3/projects/b2f19da78ef54565bf618add8926371e` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:30.498871 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742ba4c-beca-4786-8388-e003a64204e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2f19da78ef54565bf618add8926371e)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:30.501367 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742ba4c-beca-4786-8388-e003a64204e5 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:30.559929 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6742ba4c-beca-4786-8388-e003a64204e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2f19da78ef54565bf618add8926371e', 'request_id': 'req-6742ba4c-beca-4786-8388-e003a64204e5'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:30.573199 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6742ba4c-beca-4786-8388-e003a64204e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2f19da78ef54565bf618add8926371e', 'request_id': 'req-6742ba4c-beca-4786-8388-e003a64204e5'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:30.580433 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6742ba4c-beca-4786-8388-e003a64204e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2f19da78ef54565bf618add8926371e', 'request_id': 'req-6742ba4c-beca-4786-8388-e003a64204e5'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:30.593452 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6742ba4c-beca-4786-8388-e003a64204e5 admin admin] The token cache is being invalidate because project b2f19da78ef54565bf618add8926371e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:17:30.594527 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6742ba4c-beca-4786-8388-e003a64204e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:30.595449 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6742ba4c-beca-4786-8388-e003a64204e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:30.598006 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1435/2867] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:17:30 2022] DELETE /identity/v3/projects/b2f19da78ef54565bf618add8926371e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:30.604564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1c974-da74-4a77-8f31-2db219702e55 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:30.604564 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1c974-da74-4a77-8f31-2db219702e55 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:30.604811 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1c974-da74-4a77-8f31-2db219702e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:30.626314 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-55c1c974-da74-4a77-8f31-2db219702e55 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:30.627499 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-55c1c974-da74-4a77-8f31-2db219702e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:30.658743 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:30.701544 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1433/2868] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:17:30.710391 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db806574-37c5-4402-ab8b-e101fd56bd24 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:30.710682 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db806574-37c5-4402-ab8b-e101fd56bd24 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:30.710961 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db806574-37c5-4402-ab8b-e101fd56bd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:30.739798 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-db806574-37c5-4402-ab8b-e101fd56bd24 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:30.741945 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-db806574-37c5-4402-ab8b-e101fd56bd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:30.758771 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1436/2869] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:17:30.760557 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1c974-da74-4a77-8f31-2db219702e55 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:30.773021 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1c974-da74-4a77-8f31-2db219702e55 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:30.780386 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8fd505-f930-4fc9-b618-7fc0fe01a0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:30.780739 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8fd505-f930-4fc9-b618-7fc0fe01a0cc admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:30.781018 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8fd505-f930-4fc9-b618-7fc0fe01a0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:30.781306 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8fd505-f930-4fc9-b618-7fc0fe01a0cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:30.782559 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8fd505-f930-4fc9-b618-7fc0fe01a0cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:30.782858 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8fd505-f930-4fc9-b618-7fc0fe01a0cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:30.784490 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8fd505-f930-4fc9-b618-7fc0fe01a0cc admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:30.808220 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1434/2870] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:30.813099 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db806574-37c5-4402-ab8b-e101fd56bd24 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:30.829197 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db806574-37c5-4402-ab8b-e101fd56bd24 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:30.838957 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a84e25-a2f6-4504-ba90-5761e0d0ab7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:30.839708 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a84e25-a2f6-4504-ba90-5761e0d0ab7e admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:30.840526 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a84e25-a2f6-4504-ba90-5761e0d0ab7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:30.841059 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a84e25-a2f6-4504-ba90-5761e0d0ab7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:30.841786 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a84e25-a2f6-4504-ba90-5761e0d0ab7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:30.858027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a84e25-a2f6-4504-ba90-5761e0d0ab7e admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:30.889946 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1437/2871] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:17:30 2022] POST /identity/v3/projects => generated 371 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:17:30.897598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8fd505-f930-4fc9-b618-7fc0fe01a0cc admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:30.924867 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8fd505-f930-4fc9-b618-7fc0fe01a0cc admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:31.021161 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c509052-734c-4412-9ae5-fdefedb19f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:31.022095 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c509052-734c-4412-9ae5-fdefedb19f19 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:31.022373 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c509052-734c-4412-9ae5-fdefedb19f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:31.022647 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c509052-734c-4412-9ae5-fdefedb19f19 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:31.023301 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c509052-734c-4412-9ae5-fdefedb19f19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:31.025325 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c509052-734c-4412-9ae5-fdefedb19f19 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:31.052309 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1435/2872] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:17:30 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:17:31.056296 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a84e25-a2f6-4504-ba90-5761e0d0ab7e admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:31.067882 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a84e25-a2f6-4504-ba90-5761e0d0ab7e admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:31.073030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d0042b-f5e1-437f-8afa-60d83663c519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:31.073659 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0042b-f5e1-437f-8afa-60d83663c519 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:31.073947 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0042b-f5e1-437f-8afa-60d83663c519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:31.074223 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0042b-f5e1-437f-8afa-60d83663c519 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:31.074898 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d0042b-f5e1-437f-8afa-60d83663c519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:31.076566 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d0042b-f5e1-437f-8afa-60d83663c519 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:31.092762 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1438/2873] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:17:31.099240 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c509052-734c-4412-9ae5-fdefedb19f19 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:31.109235 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c509052-734c-4412-9ae5-fdefedb19f19 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:31.114014 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639c20e-b3f2-442d-b88c-e1cd87c29678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:31.114620 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639c20e-b3f2-442d-b88c-e1cd87c29678 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:31.114896 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639c20e-b3f2-442d-b88c-e1cd87c29678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:31.115165 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639c20e-b3f2-442d-b88c-e1cd87c29678 admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:31.115888 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639c20e-b3f2-442d-b88c-e1cd87c29678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:31.117461 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639c20e-b3f2-442d-b88c-e1cd87c29678 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:31.129110 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1436/2874] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:17:31.133274 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d0042b-f5e1-437f-8afa-60d83663c519 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:31.145747 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d0042b-f5e1-437f-8afa-60d83663c519 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:31.150382 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e0e1d-e1af-4169-b04e-0f47d8304cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:31.151048 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e0e1d-e1af-4169-b04e-0f47d8304cb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:31.151327 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e0e1d-e1af-4169-b04e-0f47d8304cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:31.151642 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e0e1d-e1af-4169-b04e-0f47d8304cb7 admin admin] PATH_INFO: `/v3/projects/eee31d34b2724ef085fcbb59b70552fd/users/4729dd07c69a40e29b32522bc3e46e1c/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:31.170253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e0e1d-e1af-4169-b04e-0f47d8304cb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eee31d34b2724ef085fcbb59b70552fd, user_id=4729dd07c69a40e29b32522bc3e46e1c, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:31.174637 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e0e1d-e1af-4169-b04e-0f47d8304cb7 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:31.188981 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1439/2875] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:17:31 2022] PUT /identity/v3/projects/eee31d34b2724ef085fcbb59b70552fd/users/4729dd07c69a40e29b32522bc3e46e1c/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:31.200404 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3e596-76ef-4cf1-add0-19f65d2b6bff None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:31.200703 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3e596-76ef-4cf1-add0-19f65d2b6bff None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:31.200976 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3e596-76ef-4cf1-add0-19f65d2b6bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:31.229256 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-7fb3e596-76ef-4cf1-add0-19f65d2b6bff None None] MFA Rules not processed for user `4729dd07c69a40e29b32522bc3e46e1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:31.230530 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7fb3e596-76ef-4cf1-add0-19f65d2b6bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:31.301327 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1437/2876] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:17:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:17:31.312352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e0e1d-e1af-4169-b04e-0f47d8304cb7 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:31.314294 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-844e0e1d-e1af-4169-b04e-0f47d8304cb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:31.363479 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:31.376251 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e0e1d-e1af-4169-b04e-0f47d8304cb7 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:31.379869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e014215-4faf-4c49-9516-dca296358bb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:31.380584 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e014215-4faf-4c49-9516-dca296358bb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:31.380785 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e014215-4faf-4c49-9516-dca296358bb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:31.381063 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e014215-4faf-4c49-9516-dca296358bb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:31.389775 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e014215-4faf-4c49-9516-dca296358bb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:31.391895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e014215-4faf-4c49-9516-dca296358bb5 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:31.400597 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1440/2877] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:17:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:17:31.429285 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb3e596-76ef-4cf1-add0-19f65d2b6bff None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:31.458930 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:31.495352 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb3e596-76ef-4cf1-add0-19f65d2b6bff None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:31.501590 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508ee88-8dbc-4c7a-bc36-6d796fcb3bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:31.502530 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508ee88-8dbc-4c7a-bc36-6d796fcb3bf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:31.502961 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508ee88-8dbc-4c7a-bc36-6d796fcb3bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:31.503261 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508ee88-8dbc-4c7a-bc36-6d796fcb3bf8 admin admin] PATH_INFO: `/v3/users/4729dd07c69a40e29b32522bc3e46e1c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:31.504939 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508ee88-8dbc-4c7a-bc36-6d796fcb3bf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4729dd07c69a40e29b32522bc3e46e1c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:31.507237 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508ee88-8dbc-4c7a-bc36-6d796fcb3bf8 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:31.556057 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8508ee88-8dbc-4c7a-bc36-6d796fcb3bf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4729dd07c69a40e29b32522bc3e46e1c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:31.556633 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8508ee88-8dbc-4c7a-bc36-6d796fcb3bf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4729dd07c69a40e29b32522bc3e46e1c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:31.566715 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8508ee88-8dbc-4c7a-bc36-6d796fcb3bf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4729dd07c69a40e29b32522bc3e46e1c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:31.567376 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8508ee88-8dbc-4c7a-bc36-6d796fcb3bf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4729dd07c69a40e29b32522bc3e46e1c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:31.574721 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-8508ee88-8dbc-4c7a-bc36-6d796fcb3bf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4729dd07c69a40e29b32522bc3e46e1c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:31.596652 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1438/2878] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:17:31 2022] DELETE /identity/v3/users/4729dd07c69a40e29b32522bc3e46e1c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:31.601193 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e022f-b50c-4c90-8fde-0d5395358233 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:31.601486 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e022f-b50c-4c90-8fde-0d5395358233 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:31.601766 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e022f-b50c-4c90-8fde-0d5395358233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:31.621428 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-2f4e022f-b50c-4c90-8fde-0d5395358233 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:31.622635 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2f4e022f-b50c-4c90-8fde-0d5395358233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:31.683629 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1441/2879] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:17:31.695830 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508ee88-8dbc-4c7a-bc36-6d796fcb3bf8 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:31.697255 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-8508ee88-8dbc-4c7a-bc36-6d796fcb3bf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:31.757749 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508ee88-8dbc-4c7a-bc36-6d796fcb3bf8 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:31.761926 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca38890-3f03-4980-be50-f3050a87511e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:31.762544 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca38890-3f03-4980-be50-f3050a87511e service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:31.762825 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca38890-3f03-4980-be50-f3050a87511e service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:31.763094 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca38890-3f03-4980-be50-f3050a87511e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:31.772752 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca38890-3f03-4980-be50-f3050a87511e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:31.774306 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca38890-3f03-4980-be50-f3050a87511e service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:31.786693 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1439/2880] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:17:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:17:32.087727 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e022f-b50c-4c90-8fde-0d5395358233 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:32.089362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-2f4e022f-b50c-4c90-8fde-0d5395358233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:32.110166 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e022f-b50c-4c90-8fde-0d5395358233 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:32.116431 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132852d-dd0e-4cc2-ab65-8d6f228000e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:32.117315 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132852d-dd0e-4cc2-ab65-8d6f228000e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:32.117595 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132852d-dd0e-4cc2-ab65-8d6f228000e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:32.117873 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132852d-dd0e-4cc2-ab65-8d6f228000e2 admin admin] PATH_INFO: `/v3/projects/eee31d34b2724ef085fcbb59b70552fd` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:32.119031 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132852d-dd0e-4cc2-ab65-8d6f228000e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eee31d34b2724ef085fcbb59b70552fd)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:32.121027 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132852d-dd0e-4cc2-ab65-8d6f228000e2 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:32.179114 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2132852d-dd0e-4cc2-ab65-8d6f228000e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eee31d34b2724ef085fcbb59b70552fd', 'request_id': 'req-2132852d-dd0e-4cc2-ab65-8d6f228000e2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:32.190457 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2132852d-dd0e-4cc2-ab65-8d6f228000e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eee31d34b2724ef085fcbb59b70552fd', 'request_id': 'req-2132852d-dd0e-4cc2-ab65-8d6f228000e2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:32.197226 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2132852d-dd0e-4cc2-ab65-8d6f228000e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eee31d34b2724ef085fcbb59b70552fd', 'request_id': 'req-2132852d-dd0e-4cc2-ab65-8d6f228000e2'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:32.208867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2132852d-dd0e-4cc2-ab65-8d6f228000e2 admin admin] The token cache is being invalidate because project eee31d34b2724ef085fcbb59b70552fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:17:32.209159 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2132852d-dd0e-4cc2-ab65-8d6f228000e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:32.209991 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-2132852d-dd0e-4cc2-ab65-8d6f228000e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:32.211489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1442/2881] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:17:32 2022] DELETE /identity/v3/projects/eee31d34b2724ef085fcbb59b70552fd => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:32.216477 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9854fb-e7fb-4340-935b-719ff0eecc60 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:32.216748 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9854fb-e7fb-4340-935b-719ff0eecc60 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:32.217028 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9854fb-e7fb-4340-935b-719ff0eecc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:32.246176 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-ad9854fb-e7fb-4340-935b-719ff0eecc60 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:32.247402 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ad9854fb-e7fb-4340-935b-719ff0eecc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:32.304848 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:32.322850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1440/2882] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:17:32.330591 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780252f-9362-425b-919c-8f988fe6e00e None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:32.330887 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780252f-9362-425b-919c-8f988fe6e00e None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:32.331170 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780252f-9362-425b-919c-8f988fe6e00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:32.353497 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-a780252f-9362-425b-919c-8f988fe6e00e None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:32.354903 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a780252f-9362-425b-919c-8f988fe6e00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:32.363447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1443/2883] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:17:32.367797 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9854fb-e7fb-4340-935b-719ff0eecc60 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:32.378695 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9854fb-e7fb-4340-935b-719ff0eecc60 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:32.383444 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98df224-0ca5-438a-9140-36b38757aebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:32.384381 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98df224-0ca5-438a-9140-36b38757aebd admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:32.384669 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98df224-0ca5-438a-9140-36b38757aebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:32.385207 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98df224-0ca5-438a-9140-36b38757aebd admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:32.385862 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98df224-0ca5-438a-9140-36b38757aebd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:32.386165 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98df224-0ca5-438a-9140-36b38757aebd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:32.387832 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98df224-0ca5-438a-9140-36b38757aebd admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:32.401533 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1441/2884] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:32.405859 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780252f-9362-425b-919c-8f988fe6e00e None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:32.416675 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780252f-9362-425b-919c-8f988fe6e00e None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:32.421681 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb36526-bf3d-4b59-b205-1928525844ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:32.422870 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb36526-bf3d-4b59-b205-1928525844ca admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:32.423630 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb36526-bf3d-4b59-b205-1928525844ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:32.423924 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb36526-bf3d-4b59-b205-1928525844ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:32.424617 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb36526-bf3d-4b59-b205-1928525844ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:32.426563 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb36526-bf3d-4b59-b205-1928525844ca admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:32.455925 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1444/2885] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:17:32 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:17:32.460317 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98df224-0ca5-438a-9140-36b38757aebd admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:32.471003 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98df224-0ca5-438a-9140-36b38757aebd admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:32.533608 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d12c00-7af5-403a-851f-336b40810367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:32.534245 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d12c00-7af5-403a-851f-336b40810367 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:32.534521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d12c00-7af5-403a-851f-336b40810367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:32.534797 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d12c00-7af5-403a-851f-336b40810367 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:32.535475 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d12c00-7af5-403a-851f-336b40810367 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:32.537521 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d12c00-7af5-403a-851f-336b40810367 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:32.563615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1442/2886] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:17:32 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:17:32.565398 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb36526-bf3d-4b59-b205-1928525844ca admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:32.575893 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb36526-bf3d-4b59-b205-1928525844ca admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:32.580869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f541c7a-d66a-4f4b-86ba-7d82ca2d1eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:32.581650 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f541c7a-d66a-4f4b-86ba-7d82ca2d1eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:32.582604 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f541c7a-d66a-4f4b-86ba-7d82ca2d1eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:32.582895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f541c7a-d66a-4f4b-86ba-7d82ca2d1eb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:32.583585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f541c7a-d66a-4f4b-86ba-7d82ca2d1eb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:32.585253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f541c7a-d66a-4f4b-86ba-7d82ca2d1eb6 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:32.597986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1445/2887] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:17:32.602044 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d12c00-7af5-403a-851f-336b40810367 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:32.614602 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d12c00-7af5-403a-851f-336b40810367 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:32.619854 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fca284-eca0-40d7-b7d2-67e02ab5f36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:32.620480 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fca284-eca0-40d7-b7d2-67e02ab5f36d admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:32.620749 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fca284-eca0-40d7-b7d2-67e02ab5f36d admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:32.621028 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fca284-eca0-40d7-b7d2-67e02ab5f36d admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:32.621684 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fca284-eca0-40d7-b7d2-67e02ab5f36d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:32.623297 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fca284-eca0-40d7-b7d2-67e02ab5f36d admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:32.637947 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1443/2888] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:17:32.641755 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f541c7a-d66a-4f4b-86ba-7d82ca2d1eb6 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:32.652489 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f541c7a-d66a-4f4b-86ba-7d82ca2d1eb6 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:32.657270 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a34cdc-b62d-46ba-9036-8e39565c0951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:32.658140 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a34cdc-b62d-46ba-9036-8e39565c0951 admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:32.658279 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a34cdc-b62d-46ba-9036-8e39565c0951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:32.658577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a34cdc-b62d-46ba-9036-8e39565c0951 admin admin] PATH_INFO: `/v3/projects/3569e84802cc4c82bc6238ffd16fef92/users/cc01596244b8454eb7c1508e36ba7c8c/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:32.675264 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a34cdc-b62d-46ba-9036-8e39565c0951 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3569e84802cc4c82bc6238ffd16fef92, user_id=cc01596244b8454eb7c1508e36ba7c8c, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:32.677990 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a34cdc-b62d-46ba-9036-8e39565c0951 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:32.693416 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1446/2889] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:17:32 2022] PUT /identity/v3/projects/3569e84802cc4c82bc6238ffd16fef92/users/cc01596244b8454eb7c1508e36ba7c8c/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:32.704026 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398d0c9-5774-40b1-81c5-5bf770b44cf9 None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:32.704328 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398d0c9-5774-40b1-81c5-5bf770b44cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:32.704620 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398d0c9-5774-40b1-81c5-5bf770b44cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:32.733554 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-5398d0c9-5774-40b1-81c5-5bf770b44cf9 None None] MFA Rules not processed for user `cc01596244b8454eb7c1508e36ba7c8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:32.735077 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-5398d0c9-5774-40b1-81c5-5bf770b44cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:32.807042 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1444/2890] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:17:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:17:32.818619 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a34cdc-b62d-46ba-9036-8e39565c0951 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:32.820372 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-11a34cdc-b62d-46ba-9036-8e39565c0951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:32.827020 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:32.894006 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a34cdc-b62d-46ba-9036-8e39565c0951 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:32.897352 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838977a-5c38-41fd-ba55-dcb7fb3e7c69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:32.897987 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838977a-5c38-41fd-ba55-dcb7fb3e7c69 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:32.898265 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838977a-5c38-41fd-ba55-dcb7fb3e7c69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:32.898558 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838977a-5c38-41fd-ba55-dcb7fb3e7c69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:32.909566 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7838977a-5c38-41fd-ba55-dcb7fb3e7c69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:32.911248 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7838977a-5c38-41fd-ba55-dcb7fb3e7c69 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:32.927649 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1447/2891] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:17:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:17:42.803524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=77,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:42.922253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=24 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:43.543905 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398d0c9-5774-40b1-81c5-5bf770b44cf9 None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:43.610409 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398d0c9-5774-40b1-81c5-5bf770b44cf9 None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:43.615091 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7c364-8f7c-4b63-90e6-bc6de684d3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:43.615810 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7c364-8f7c-4b63-90e6-bc6de684d3f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:43.616089 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7c364-8f7c-4b63-90e6-bc6de684d3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:43.616360 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7c364-8f7c-4b63-90e6-bc6de684d3f0 admin admin] PATH_INFO: `/v3/users/cc01596244b8454eb7c1508e36ba7c8c` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:43.617467 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f7c364-8f7c-4b63-90e6-bc6de684d3f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc01596244b8454eb7c1508e36ba7c8c)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:43.619365 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f7c364-8f7c-4b63-90e6-bc6de684d3f0 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:43.671669 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-55f7c364-8f7c-4b63-90e6-bc6de684d3f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc01596244b8454eb7c1508e36ba7c8c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:43.672463 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-55f7c364-8f7c-4b63-90e6-bc6de684d3f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc01596244b8454eb7c1508e36ba7c8c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:43.688591 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-55f7c364-8f7c-4b63-90e6-bc6de684d3f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc01596244b8454eb7c1508e36ba7c8c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:43.689351 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-55f7c364-8f7c-4b63-90e6-bc6de684d3f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc01596244b8454eb7c1508e36ba7c8c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:43.697372 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-55f7c364-8f7c-4b63-90e6-bc6de684d3f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc01596244b8454eb7c1508e36ba7c8c'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:43.715313 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1445/2892] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:17:43 2022] DELETE /identity/v3/users/cc01596244b8454eb7c1508e36ba7c8c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:43.719791 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb762a-51e7-4752-9d9c-fbf81877a0a0 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:43.720082 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb762a-51e7-4752-9d9c-fbf81877a0a0 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:43.720365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb762a-51e7-4752-9d9c-fbf81877a0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:43.739330 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-e1eb762a-51e7-4752-9d9c-fbf81877a0a0 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:43.740678 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e1eb762a-51e7-4752-9d9c-fbf81877a0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:43.802689 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1448/2893] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:17:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:17:43.812771 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7c364-8f7c-4b63-90e6-bc6de684d3f0 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:43.814197 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-55f7c364-8f7c-4b63-90e6-bc6de684d3f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:43.876642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7c364-8f7c-4b63-90e6-bc6de684d3f0 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:43.881154 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277247c-5b3e-470a-afac-48e38ea8f2a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:43.881759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277247c-5b3e-470a-afac-48e38ea8f2a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:43.882040 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277247c-5b3e-470a-afac-48e38ea8f2a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:43.882311 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277247c-5b3e-470a-afac-48e38ea8f2a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:43.891446 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8277247c-5b3e-470a-afac-48e38ea8f2a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:43.892905 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8277247c-5b3e-470a-afac-48e38ea8f2a6 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:43.902380 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1446/2894] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:17:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:17:44.177316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb762a-51e7-4752-9d9c-fbf81877a0a0 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:44.179081 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e1eb762a-51e7-4752-9d9c-fbf81877a0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:44.196370 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb762a-51e7-4752-9d9c-fbf81877a0a0 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:44.208092 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ab3606-572b-4ad5-a59b-a80c0651498a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:44.208761 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab3606-572b-4ad5-a59b-a80c0651498a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:44.209044 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab3606-572b-4ad5-a59b-a80c0651498a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:44.209316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab3606-572b-4ad5-a59b-a80c0651498a admin admin] PATH_INFO: `/v3/projects/3569e84802cc4c82bc6238ffd16fef92` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:44.210447 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab3606-572b-4ad5-a59b-a80c0651498a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3569e84802cc4c82bc6238ffd16fef92)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:44.212389 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab3606-572b-4ad5-a59b-a80c0651498a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:44.263960 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d6ab3606-572b-4ad5-a59b-a80c0651498a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3569e84802cc4c82bc6238ffd16fef92', 'request_id': 'req-d6ab3606-572b-4ad5-a59b-a80c0651498a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:44.271757 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d6ab3606-572b-4ad5-a59b-a80c0651498a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3569e84802cc4c82bc6238ffd16fef92', 'request_id': 'req-d6ab3606-572b-4ad5-a59b-a80c0651498a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:44.277399 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d6ab3606-572b-4ad5-a59b-a80c0651498a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3569e84802cc4c82bc6238ffd16fef92', 'request_id': 'req-d6ab3606-572b-4ad5-a59b-a80c0651498a'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:44.289077 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d6ab3606-572b-4ad5-a59b-a80c0651498a admin admin] The token cache is being invalidate because project 3569e84802cc4c82bc6238ffd16fef92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81418) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:17:44.289366 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d6ab3606-572b-4ad5-a59b-a80c0651498a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:44.290005 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-d6ab3606-572b-4ad5-a59b-a80c0651498a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:17:44.291527 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1449/2895] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:17:44 2022] DELETE /identity/v3/projects/3569e84802cc4c82bc6238ffd16fef92 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:17:44.296716 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef1266-e7d8-4d2f-8bae-0b7650bf8e0e None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:44.297012 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef1266-e7d8-4d2f-8bae-0b7650bf8e0e None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:44.297296 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef1266-e7d8-4d2f-8bae-0b7650bf8e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:44.318958 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-d9ef1266-e7d8-4d2f-8bae-0b7650bf8e0e None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:44.320209 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d9ef1266-e7d8-4d2f-8bae-0b7650bf8e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:44.337000 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:44.382890 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1447/2896] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:17:44.447293 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf3770-ccfc-4a65-87ce-35d5bbfda2e5 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:44.447612 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf3770-ccfc-4a65-87ce-35d5bbfda2e5 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:44.447895 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf3770-ccfc-4a65-87ce-35d5bbfda2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:44.468387 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-9cdf3770-ccfc-4a65-87ce-35d5bbfda2e5 None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:44.469643 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-9cdf3770-ccfc-4a65-87ce-35d5bbfda2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:44.479808 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1450/2897] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:17:44.484190 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef1266-e7d8-4d2f-8bae-0b7650bf8e0e None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:44.494753 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef1266-e7d8-4d2f-8bae-0b7650bf8e0e None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:44.499858 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fde42a-71c8-4743-b7c2-905cb007d883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:44.500468 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fde42a-71c8-4743-b7c2-905cb007d883 admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:44.500749 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fde42a-71c8-4743-b7c2-905cb007d883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:44.501021 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fde42a-71c8-4743-b7c2-905cb007d883 admin admin] PATH_INFO: `/v3/domains` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:44.501672 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fde42a-71c8-4743-b7c2-905cb007d883 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81417) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:44.501977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fde42a-71c8-4743-b7c2-905cb007d883 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:44.503587 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fde42a-71c8-4743-b7c2-905cb007d883 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:44.515642 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1448/2898] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 01:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:44.519986 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf3770-ccfc-4a65-87ce-35d5bbfda2e5 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:44.529028 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf3770-ccfc-4a65-87ce-35d5bbfda2e5 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:44.533634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec479a-2099-4b33-acb1-612c47ee9b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:44.534234 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec479a-2099-4b33-acb1-612c47ee9b4a admin admin] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:44.534519 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec479a-2099-4b33-acb1-612c47ee9b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:44.534795 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec479a-2099-4b33-acb1-612c47ee9b4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:44.536021 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ec479a-2099-4b33-acb1-612c47ee9b4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:44.537944 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ec479a-2099-4b33-acb1-612c47ee9b4a admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:44.567628 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1451/2899] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 01:17:44 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:17:44.572078 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fde42a-71c8-4743-b7c2-905cb007d883 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:44.582477 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fde42a-71c8-4743-b7c2-905cb007d883 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:44.639001 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9aa72-8638-4346-ad3f-f145baa8db30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:44.639641 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9aa72-8638-4346-ad3f-f145baa8db30 admin admin] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:44.639926 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9aa72-8638-4346-ad3f-f145baa8db30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:44.640202 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9aa72-8638-4346-ad3f-f145baa8db30 admin admin] PATH_INFO: `/v3/users` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:44.640842 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9aa72-8638-4346-ad3f-f145baa8db30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:44.642783 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9aa72-8638-4346-ad3f-f145baa8db30 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:44.668286 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1449/2900] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 01:17:44 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:17:44.672436 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec479a-2099-4b33-acb1-612c47ee9b4a admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:44.683011 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec479a-2099-4b33-acb1-612c47ee9b4a admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:44.687585 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a79e5-a51e-434a-8642-98e579e7ef3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:44.688193 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a79e5-a51e-434a-8642-98e579e7ef3e admin admin] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:44.688479 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a79e5-a51e-434a-8642-98e579e7ef3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:44.688755 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a79e5-a51e-434a-8642-98e579e7ef3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:44.689392 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940a79e5-a51e-434a-8642-98e579e7ef3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:44.690989 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940a79e5-a51e-434a-8642-98e579e7ef3e admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:44.700962 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1452/2901] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:17:44.705250 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9aa72-8638-4346-ad3f-f145baa8db30 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:44.717980 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9aa72-8638-4346-ad3f-f145baa8db30 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:44.723512 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26277a51-8e9b-4a48-99eb-a786f0c8ca6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:44.724265 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26277a51-8e9b-4a48-99eb-a786f0c8ca6b admin admin] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:44.724607 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26277a51-8e9b-4a48-99eb-a786f0c8ca6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:44.725089 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26277a51-8e9b-4a48-99eb-a786f0c8ca6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:44.725773 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26277a51-8e9b-4a48-99eb-a786f0c8ca6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:44.727425 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26277a51-8e9b-4a48-99eb-a786f0c8ca6b admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:44.738924 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1450/2902] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 01:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:17:44.746316 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a79e5-a51e-434a-8642-98e579e7ef3e admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:44.753320 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a79e5-a51e-434a-8642-98e579e7ef3e admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:44.758009 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e86f3-8fbc-499a-ae08-f6dd9695a42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:44.758656 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e86f3-8fbc-499a-ae08-f6dd9695a42c admin admin] REQUEST_METHOD: `PUT` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:44.758940 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e86f3-8fbc-499a-ae08-f6dd9695a42c admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:44.759218 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e86f3-8fbc-499a-ae08-f6dd9695a42c admin admin] PATH_INFO: `/v3/projects/ec8216ff972f43bfa45a803ca29478f1/users/7f224f1a1621479d862540dda526685c/roles/5d9045e45840408a92ceca71e9184ba2` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:44.774923 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e86f3-8fbc-499a-ae08-f6dd9695a42c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8216ff972f43bfa45a803ca29478f1, user_id=7f224f1a1621479d862540dda526685c, role_id=5d9045e45840408a92ceca71e9184ba2)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:44.777700 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e86f3-8fbc-499a-ae08-f6dd9695a42c admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:44.791264 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1453/2903] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 01:17:44 2022] PUT /identity/v3/projects/ec8216ff972f43bfa45a803ca29478f1/users/7f224f1a1621479d862540dda526685c/roles/5d9045e45840408a92ceca71e9184ba2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:44.801986 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46656e13-e2fb-42f3-8575-87ee678415bd None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:44.802276 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46656e13-e2fb-42f3-8575-87ee678415bd None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:44.802604 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46656e13-e2fb-42f3-8575-87ee678415bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:44.833382 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-46656e13-e2fb-42f3-8575-87ee678415bd None None] MFA Rules not processed for user `7f224f1a1621479d862540dda526685c`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:44.834619 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-46656e13-e2fb-42f3-8575-87ee678415bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:44.899301 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1451/2904] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:17:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:17:44.909975 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e86f3-8fbc-499a-ae08-f6dd9695a42c admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:44.912313 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-059e86f3-8fbc-499a-ae08-f6dd9695a42c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:44.933574 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:44.969725 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e86f3-8fbc-499a-ae08-f6dd9695a42c admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:44.973566 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061f348-028b-4a8c-b4ad-3d9fb9ac0265 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:44.974161 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061f348-028b-4a8c-b4ad-3d9fb9ac0265 service cinder] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:44.974440 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061f348-028b-4a8c-b4ad-3d9fb9ac0265 service cinder] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:44.974715 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061f348-028b-4a8c-b4ad-3d9fb9ac0265 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:44.982739 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061f348-028b-4a8c-b4ad-3d9fb9ac0265 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:44.984201 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061f348-028b-4a8c-b4ad-3d9fb9ac0265 service cinder] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:44.994465 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1454/2905] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:17:44 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:17:54.896278 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:54.988836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=17 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:58.518244 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46656e13-e2fb-42f3-8575-87ee678415bd None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:58.519659 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-46656e13-e2fb-42f3-8575-87ee678415bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:58.590934 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46656e13-e2fb-42f3-8575-87ee678415bd None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:58.595916 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd675d-c7de-463a-a7fd-97e2dc570f9a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18798838de284c84b5b838b3530a2c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:58.596524 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd675d-c7de-463a-a7fd-97e2dc570f9a service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:58.596807 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd675d-c7de-463a-a7fd-97e2dc570f9a service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:58.597079 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd675d-c7de-463a-a7fd-97e2dc570f9a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:58.598779 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d1cd675d-c7de-463a-a7fd-97e2dc570f9a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:58.610176 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd675d-c7de-463a-a7fd-97e2dc570f9a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:58.611779 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd675d-c7de-463a-a7fd-97e2dc570f9a service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:58.621746 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1452/2906] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:17:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:18:08.617467 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=32 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:12.477955 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061f348-028b-4a8c-b4ad-3d9fb9ac0265 service cinder] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:12.479549 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-b061f348-028b-4a8c-b4ad-3d9fb9ac0265 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:12.537229 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061f348-028b-4a8c-b4ad-3d9fb9ac0265 service cinder] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:12.540788 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8c494-7de9-4747-be5f-4dcb031cd73b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:12.541458 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8c494-7de9-4747-be5f-4dcb031cd73b service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:12.541732 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8c494-7de9-4747-be5f-4dcb031cd73b service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:12.542002 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8c494-7de9-4747-be5f-4dcb031cd73b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:12.543787 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a9a8c494-7de9-4747-be5f-4dcb031cd73b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:12.553461 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8c494-7de9-4747-be5f-4dcb031cd73b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:12.554978 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8c494-7de9-4747-be5f-4dcb031cd73b service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:12.563929 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1455/2907] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:18:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:18:13.024146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd675d-c7de-463a-a7fd-97e2dc570f9a service placement] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:13.035360 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd675d-c7de-463a-a7fd-97e2dc570f9a service placement] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:13.040326 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b3e79-37a1-4484-aed5-fd8067f10798 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18798838de284c84b5b838b3530a2c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:13.041666 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b3e79-37a1-4484-aed5-fd8067f10798 service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:13.041946 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b3e79-37a1-4484-aed5-fd8067f10798 service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:13.042219 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b3e79-37a1-4484-aed5-fd8067f10798 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:13.044095 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-095b3e79-37a1-4484-aed5-fd8067f10798 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:13.056093 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b3e79-37a1-4484-aed5-fd8067f10798 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:13.057634 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b3e79-37a1-4484-aed5-fd8067f10798 service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:13.068158 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1453/2908] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:18:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:18:15.666483 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8c494-7de9-4747-be5f-4dcb031cd73b service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:15.745180 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8c494-7de9-4747-be5f-4dcb031cd73b service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:15.750045 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48287445-2e96-461e-b1bc-0facd9d4f9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:15.750813 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48287445-2e96-461e-b1bc-0facd9d4f9e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:15.751158 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48287445-2e96-461e-b1bc-0facd9d4f9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:15.751502 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48287445-2e96-461e-b1bc-0facd9d4f9e5 admin admin] PATH_INFO: `/v3/users/7f224f1a1621479d862540dda526685c` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:15.752764 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48287445-2e96-461e-b1bc-0facd9d4f9e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f224f1a1621479d862540dda526685c)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:15.754814 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48287445-2e96-461e-b1bc-0facd9d4f9e5 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:15.805396 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-48287445-2e96-461e-b1bc-0facd9d4f9e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f224f1a1621479d862540dda526685c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:18:15.806082 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-48287445-2e96-461e-b1bc-0facd9d4f9e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f224f1a1621479d862540dda526685c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:18:15.814220 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-48287445-2e96-461e-b1bc-0facd9d4f9e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f224f1a1621479d862540dda526685c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:18:15.821395 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-48287445-2e96-461e-b1bc-0facd9d4f9e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f224f1a1621479d862540dda526685c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:18:15.822063 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-48287445-2e96-461e-b1bc-0facd9d4f9e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f224f1a1621479d862540dda526685c'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:18:15.839461 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1456/2909] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:18:15 2022] DELETE /identity/v3/users/7f224f1a1621479d862540dda526685c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:18:15.844046 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea2719-46b2-487c-a94d-7ba15ef482ad None None] REQUEST_METHOD: `POST` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:15.844537 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea2719-46b2-487c-a94d-7ba15ef482ad None None] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:15.844666 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea2719-46b2-487c-a94d-7ba15ef482ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:15.864172 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.auth.core [None req-cdea2719-46b2-487c-a94d-7ba15ef482ad None None] MFA Rules not processed for user `9dea2911925e405b852c2fe0f9bee81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81417) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:15.865661 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cdea2719-46b2-487c-a94d-7ba15ef482ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:15.933654 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1454/2910] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:18:15.948497 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48287445-2e96-461e-b1bc-0facd9d4f9e5 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:15.950192 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-48287445-2e96-461e-b1bc-0facd9d4f9e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:15.977816 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:16.011859 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48287445-2e96-461e-b1bc-0facd9d4f9e5 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:16.015471 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff276685-13a9-42c1-b281-63b8c97fd0f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:16.016120 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff276685-13a9-42c1-b281-63b8c97fd0f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:16.016397 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff276685-13a9-42c1-b281-63b8c97fd0f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:16.016668 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff276685-13a9-42c1-b281-63b8c97fd0f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:16.027450 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff276685-13a9-42c1-b281-63b8c97fd0f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:16.029132 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff276685-13a9-42c1-b281-63b8c97fd0f1 service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:16.039061 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1457/2911] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:18:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:18:16.385378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea2719-46b2-487c-a94d-7ba15ef482ad None None] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:16.387368 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-cdea2719-46b2-487c-a94d-7ba15ef482ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:16.406085 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea2719-46b2-487c-a94d-7ba15ef482ad None None] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:16.411478 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e44af-3bad-40e8-a328-792cd26583f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:16.412106 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e44af-3bad-40e8-a328-792cd26583f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:16.412387 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e44af-3bad-40e8-a328-792cd26583f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:16.412668 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e44af-3bad-40e8-a328-792cd26583f9 admin admin] PATH_INFO: `/v3/projects/ec8216ff972f43bfa45a803ca29478f1` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:16.413785 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766e44af-3bad-40e8-a328-792cd26583f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec8216ff972f43bfa45a803ca29478f1)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:16.415820 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766e44af-3bad-40e8-a328-792cd26583f9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:16.466454 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-766e44af-3bad-40e8-a328-792cd26583f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec8216ff972f43bfa45a803ca29478f1', 'request_id': 'req-766e44af-3bad-40e8-a328-792cd26583f9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:18:16.475038 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-766e44af-3bad-40e8-a328-792cd26583f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec8216ff972f43bfa45a803ca29478f1', 'request_id': 'req-766e44af-3bad-40e8-a328-792cd26583f9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:18:16.485085 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-766e44af-3bad-40e8-a328-792cd26583f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec8216ff972f43bfa45a803ca29478f1', 'request_id': 'req-766e44af-3bad-40e8-a328-792cd26583f9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:18:16.497358 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-766e44af-3bad-40e8-a328-792cd26583f9 admin admin] The token cache is being invalidate because project ec8216ff972f43bfa45a803ca29478f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:18:16.497645 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-766e44af-3bad-40e8-a328-792cd26583f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:18:16.498317 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-766e44af-3bad-40e8-a328-792cd26583f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:18:16.500005 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1455/2912] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:18:16 2022] DELETE /identity/v3/projects/ec8216ff972f43bfa45a803ca29478f1 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:18:17.433320 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be482ec6-82e3-40c1-a06a-2d720f84b585 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:17.433867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be482ec6-82e3-40c1-a06a-2d720f84b585 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:17.433867 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be482ec6-82e3-40c1-a06a-2d720f84b585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:17.452655 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-be482ec6-82e3-40c1-a06a-2d720f84b585 None None] MFA Rules not processed for user `b1494511685b47519f4aa0674009fbfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:17.453953 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-be482ec6-82e3-40c1-a06a-2d720f84b585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:17.514965 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1458/2913] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:18:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:18:26.493403 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:27.506885 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=41 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:28.001264 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e44af-3bad-40e8-a328-792cd26583f9 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:28.002697 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-766e44af-3bad-40e8-a328-792cd26583f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:28.078330 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e44af-3bad-40e8-a328-792cd26583f9 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:28.083890 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306b2dd-1b10-46d9-aacc-ddfb3384e917 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:28.084535 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306b2dd-1b10-46d9-aacc-ddfb3384e917 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:28.084815 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306b2dd-1b10-46d9-aacc-ddfb3384e917 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:28.085090 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306b2dd-1b10-46d9-aacc-ddfb3384e917 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:28.086812 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d306b2dd-1b10-46d9-aacc-ddfb3384e917 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:28.099513 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d306b2dd-1b10-46d9-aacc-ddfb3384e917 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:28.101133 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d306b2dd-1b10-46d9-aacc-ddfb3384e917 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:28.113335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1456/2914] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:18:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:18:32.416672 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be482ec6-82e3-40c1-a06a-2d720f84b585 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:32.418293 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-be482ec6-82e3-40c1-a06a-2d720f84b585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:32.480926 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be482ec6-82e3-40c1-a06a-2d720f84b585 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:32.484936 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657b55e-b8bc-4c4f-b85c-c703a9e7809c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:32.485554 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657b55e-b8bc-4c4f-b85c-c703a9e7809c service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:32.485830 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657b55e-b8bc-4c4f-b85c-c703a9e7809c service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:32.486105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4657b55e-b8bc-4c4f-b85c-c703a9e7809c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:32.487876 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4657b55e-b8bc-4c4f-b85c-c703a9e7809c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:32.553237 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657b55e-b8bc-4c4f-b85c-c703a9e7809c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:32.554689 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4657b55e-b8bc-4c4f-b85c-c703a9e7809c service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:32.564631 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1459/2915] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:18:32 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:18:36.084199 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306b2dd-1b10-46d9-aacc-ddfb3384e917 service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:36.085710 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-d306b2dd-1b10-46d9-aacc-ddfb3384e917 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:36.097811 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306b2dd-1b10-46d9-aacc-ddfb3384e917 service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:36.101461 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383af6fa-d298-45f4-a116-e745f4f19f9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:36.102061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383af6fa-d298-45f4-a116-e745f4f19f9b service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:36.102342 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383af6fa-d298-45f4-a116-e745f4f19f9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:36.102615 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383af6fa-d298-45f4-a116-e745f4f19f9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:36.104901 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-383af6fa-d298-45f4-a116-e745f4f19f9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:36.169151 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383af6fa-d298-45f4-a116-e745f4f19f9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:36.170593 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383af6fa-d298-45f4-a116-e745f4f19f9b service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:36.180081 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1457/2916] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:18:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:18:37.959354 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657b55e-b8bc-4c4f-b85c-c703a9e7809c service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:37.972105 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4657b55e-b8bc-4c4f-b85c-c703a9e7809c service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:37.977779 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215c440-0933-4251-985e-25514acd6508 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:37.978478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215c440-0933-4251-985e-25514acd6508 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:37.978478 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215c440-0933-4251-985e-25514acd6508 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:37.979008 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215c440-0933-4251-985e-25514acd6508 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:37.980803 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-4215c440-0933-4251-985e-25514acd6508 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:37.993207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215c440-0933-4251-985e-25514acd6508 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:37.995298 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215c440-0933-4251-985e-25514acd6508 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:38.005924 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1460/2917] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:18:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:18:46.175854 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=64 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:48.001548 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=56 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:20:48.705545 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383af6fa-d298-45f4-a116-e745f4f19f9b service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:20:48.720276 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383af6fa-d298-45f4-a116-e745f4f19f9b service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:20:48.724128 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ed8b8-4d7a-40a5-af55-aa3607a04c10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:20:48.724737 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ed8b8-4d7a-40a5-af55-aa3607a04c10 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:48.725019 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ed8b8-4d7a-40a5-af55-aa3607a04c10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:48.725291 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ed8b8-4d7a-40a5-af55-aa3607a04c10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:48.727022 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-7d1ed8b8-4d7a-40a5-af55-aa3607a04c10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:20:48.741281 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ed8b8-4d7a-40a5-af55-aa3607a04c10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:20:48.742651 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ed8b8-4d7a-40a5-af55-aa3607a04c10 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:20:48.754281 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1458/2918] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:20:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:20:49.942628 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215c440-0933-4251-985e-25514acd6508 service nova] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:20:49.951972 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215c440-0933-4251-985e-25514acd6508 service nova] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:20:49.955536 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c003c-31fb-4ac3-98a9-cf9af1f7156b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:20:49.956156 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c003c-31fb-4ac3-98a9-cf9af1f7156b service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:49.956995 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c003c-31fb-4ac3-98a9-cf9af1f7156b service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:49.957507 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c003c-31fb-4ac3-98a9-cf9af1f7156b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:49.959256 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-af1c003c-31fb-4ac3-98a9-cf9af1f7156b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:20:49.973025 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c003c-31fb-4ac3-98a9-cf9af1f7156b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:20:49.974450 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c003c-31fb-4ac3-98a9-cf9af1f7156b service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:20:49.985295 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1461/2919] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:20:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:20:57.186845 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ed8b8-4d7a-40a5-af55-aa3607a04c10 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:20:57.197770 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ed8b8-4d7a-40a5-af55-aa3607a04c10 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:20:57.202611 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea81939-0c2b-4171-93bd-233ad82c37bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:20:57.203228 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea81939-0c2b-4171-93bd-233ad82c37bd service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:57.203876 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea81939-0c2b-4171-93bd-233ad82c37bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:57.204380 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea81939-0c2b-4171-93bd-233ad82c37bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:57.206451 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-bea81939-0c2b-4171-93bd-233ad82c37bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:20:57.229051 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea81939-0c2b-4171-93bd-233ad82c37bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:20:57.230603 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea81939-0c2b-4171-93bd-233ad82c37bd service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:20:57.240601 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1459/2920] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:20:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:20:58.119541 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd8162-6067-4b04-ba40-2ef9b6258737 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:58.120052 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd8162-6067-4b04-ba40-2ef9b6258737 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:58.120159 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd8162-6067-4b04-ba40-2ef9b6258737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:58.138521 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-13dd8162-6067-4b04-ba40-2ef9b6258737 None None] MFA Rules not processed for user `b1494511685b47519f4aa0674009fbfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:20:58.140003 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-13dd8162-6067-4b04-ba40-2ef9b6258737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:20:58.147472 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1462/2921] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:20:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:21:03.344781 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea81939-0c2b-4171-93bd-233ad82c37bd service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:03.353777 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea81939-0c2b-4171-93bd-233ad82c37bd service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:03.357507 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafdca3-93f7-4a3e-a6c2-86e1d8967812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:03.358108 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafdca3-93f7-4a3e-a6c2-86e1d8967812 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:03.358393 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafdca3-93f7-4a3e-a6c2-86e1d8967812 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:03.358663 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafdca3-93f7-4a3e-a6c2-86e1d8967812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:03.360466 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-ccafdca3-93f7-4a3e-a6c2-86e1d8967812 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:21:03.370881 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccafdca3-93f7-4a3e-a6c2-86e1d8967812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:03.372348 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccafdca3-93f7-4a3e-a6c2-86e1d8967812 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:03.382331 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1460/2922] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:21:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:21:03.554929 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd8162-6067-4b04-ba40-2ef9b6258737 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:03.568845 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd8162-6067-4b04-ba40-2ef9b6258737 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:03.574496 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af5ea5-0020-4f88-a3e8-3e70a03827af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:03.575365 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af5ea5-0020-4f88-a3e8-3e70a03827af service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:03.575690 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af5ea5-0020-4f88-a3e8-3e70a03827af service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:03.575970 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af5ea5-0020-4f88-a3e8-3e70a03827af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:03.577909 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-95af5ea5-0020-4f88-a3e8-3e70a03827af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:21:03.589832 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95af5ea5-0020-4f88-a3e8-3e70a03827af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:03.591506 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95af5ea5-0020-4f88-a3e8-3e70a03827af service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:03.601399 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1463/2923] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:21:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:21:03.773552 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafdca3-93f7-4a3e-a6c2-86e1d8967812 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:03.782892 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafdca3-93f7-4a3e-a6c2-86e1d8967812 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:03.787339 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b27dff-796e-4f43-9c47-486f1c23e077 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:03.788013 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b27dff-796e-4f43-9c47-486f1c23e077 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:03.788294 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b27dff-796e-4f43-9c47-486f1c23e077 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:03.788566 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b27dff-796e-4f43-9c47-486f1c23e077 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:03.790328 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-40b27dff-796e-4f43-9c47-486f1c23e077 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:21:03.804201 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b27dff-796e-4f43-9c47-486f1c23e077 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:03.805702 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b27dff-796e-4f43-9c47-486f1c23e077 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:03.817293 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1461/2924] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:21:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:21:13.596178 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=15 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:13.811747 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=24 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:22:49.360235 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af5ea5-0020-4f88-a3e8-3e70a03827af service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:22:49.381186 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af5ea5-0020-4f88-a3e8-3e70a03827af service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:22:49.385046 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3380ef-5217-42de-861b-2aba0b94ec5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:22:49.385653 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3380ef-5217-42de-861b-2aba0b94ec5c service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:22:49.385933 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3380ef-5217-42de-861b-2aba0b94ec5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:22:49.386204 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3380ef-5217-42de-861b-2aba0b94ec5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:22:49.387971 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-7e3380ef-5217-42de-861b-2aba0b94ec5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:22:49.445758 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3380ef-5217-42de-861b-2aba0b94ec5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:22:49.447177 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3380ef-5217-42de-861b-2aba0b94ec5c service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:22:49.456286 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1464/2925] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:22:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:22:50.562768 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b27dff-796e-4f43-9c47-486f1c23e077 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:22:50.564378 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-40b27dff-796e-4f43-9c47-486f1c23e077 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:22:50.593932 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b27dff-796e-4f43-9c47-486f1c23e077 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:22:50.599952 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba14b92-0220-48eb-8e58-660c346597c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:22:50.600562 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba14b92-0220-48eb-8e58-660c346597c1 service nova] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:22:50.600844 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba14b92-0220-48eb-8e58-660c346597c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:22:50.601120 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba14b92-0220-48eb-8e58-660c346597c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:22:50.603766 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9ba14b92-0220-48eb-8e58-660c346597c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:22:50.616335 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba14b92-0220-48eb-8e58-660c346597c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:22:50.617970 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba14b92-0220-48eb-8e58-660c346597c1 service nova] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:22:50.629758 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1462/2926] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:22:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:22:54.445634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3380ef-5217-42de-861b-2aba0b94ec5c service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:22:54.457186 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3380ef-5217-42de-861b-2aba0b94ec5c service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:22:54.460863 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2cc7d-45a2-47f1-80e8-838a70683c8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:22:54.461836 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2cc7d-45a2-47f1-80e8-838a70683c8d service neutron] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:22:54.462159 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2cc7d-45a2-47f1-80e8-838a70683c8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:22:54.462436 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2cc7d-45a2-47f1-80e8-838a70683c8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:22:54.464207 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-eec2cc7d-45a2-47f1-80e8-838a70683c8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:22:54.573938 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2cc7d-45a2-47f1-80e8-838a70683c8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:22:54.575289 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2cc7d-45a2-47f1-80e8-838a70683c8d service neutron] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:22:54.593996 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1465/2927] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:22:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:22:57.662477 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba14b92-0220-48eb-8e58-660c346597c1 service nova] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:22:57.663915 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-9ba14b92-0220-48eb-8e58-660c346597c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:22:57.810644 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba14b92-0220-48eb-8e58-660c346597c1 service nova] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:22:57.815794 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a0900-efee-4a55-a349-184cce0a17cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18798838de284c84b5b838b3530a2c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:22:57.816418 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a0900-efee-4a55-a349-184cce0a17cc service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:22:57.816703 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a0900-efee-4a55-a349-184cce0a17cc service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:22:57.816981 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a0900-efee-4a55-a349-184cce0a17cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:22:57.818704 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-718a0900-efee-4a55-a349-184cce0a17cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:22:57.831413 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a0900-efee-4a55-a349-184cce0a17cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:22:57.833030 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a0900-efee-4a55-a349-184cce0a17cc service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:22:57.843168 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1463/2928] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:22:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:23:00.725715 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2cc7d-45a2-47f1-80e8-838a70683c8d service neutron] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:23:00.727811 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-eec2cc7d-45a2-47f1-80e8-838a70683c8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:00.745402 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2cc7d-45a2-47f1-80e8-838a70683c8d service neutron] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:23:00.753092 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7225ce5-4027-4294-9a4f-e0ceee12bab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:23:00.753739 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7225ce5-4027-4294-9a4f-e0ceee12bab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:00.754044 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7225ce5-4027-4294-9a4f-e0ceee12bab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:00.754321 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7225ce5-4027-4294-9a4f-e0ceee12bab0 admin admin] PATH_INFO: `/v3/users/9e0e8bab9899415c91d3b7122ab0a1d0` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:00.755731 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7225ce5-4027-4294-9a4f-e0ceee12bab0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e0e8bab9899415c91d3b7122ab0a1d0)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:23:00.759335 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7225ce5-4027-4294-9a4f-e0ceee12bab0 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:23:00.791820 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:00.812377 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a7225ce5-4027-4294-9a4f-e0ceee12bab0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e0e8bab9899415c91d3b7122ab0a1d0'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:23:00.813099 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a7225ce5-4027-4294-9a4f-e0ceee12bab0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e0e8bab9899415c91d3b7122ab0a1d0'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:23:00.821189 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a7225ce5-4027-4294-9a4f-e0ceee12bab0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e0e8bab9899415c91d3b7122ab0a1d0'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:23:00.828450 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a7225ce5-4027-4294-9a4f-e0ceee12bab0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e0e8bab9899415c91d3b7122ab0a1d0'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:23:00.829101 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-a7225ce5-4027-4294-9a4f-e0ceee12bab0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e0e8bab9899415c91d3b7122ab0a1d0'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:23:00.847404 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1466/2929] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:23:00 2022] DELETE /identity/v3/users/9e0e8bab9899415c91d3b7122ab0a1d0 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:23:01.021706 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a0900-efee-4a55-a349-184cce0a17cc service placement] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:23:01.033638 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a0900-efee-4a55-a349-184cce0a17cc service placement] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:23:01.093716 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80c12c-315c-48a4-a358-678e623c9f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:23:01.094273 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80c12c-315c-48a4-a358-678e623c9f13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:01.094554 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80c12c-315c-48a4-a358-678e623c9f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:01.094828 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80c12c-315c-48a4-a358-678e623c9f13 admin admin] PATH_INFO: `/v3/projects/b2d80d1c2758499da66a93e91f9c1311` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:01.095967 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80c12c-315c-48a4-a358-678e623c9f13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2d80d1c2758499da66a93e91f9c1311)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:23:01.097871 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80c12c-315c-48a4-a358-678e623c9f13 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:23:01.108982 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=100 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:01.146247 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0b80c12c-315c-48a4-a358-678e623c9f13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2d80d1c2758499da66a93e91f9c1311', 'request_id': 'req-0b80c12c-315c-48a4-a358-678e623c9f13'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:23:01.153598 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0b80c12c-315c-48a4-a358-678e623c9f13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2d80d1c2758499da66a93e91f9c1311', 'request_id': 'req-0b80c12c-315c-48a4-a358-678e623c9f13'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:23:01.159894 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0b80c12c-315c-48a4-a358-678e623c9f13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2d80d1c2758499da66a93e91f9c1311', 'request_id': 'req-0b80c12c-315c-48a4-a358-678e623c9f13'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:23:01.171872 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0b80c12c-315c-48a4-a358-678e623c9f13 admin admin] The token cache is being invalidate because project b2d80d1c2758499da66a93e91f9c1311 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:23:01.172177 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0b80c12c-315c-48a4-a358-678e623c9f13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:23:01.172835 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-0b80c12c-315c-48a4-a358-678e623c9f13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:23:01.174304 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1464/2930] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:23:01 2022] DELETE /identity/v3/projects/b2d80d1c2758499da66a93e91f9c1311 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:23:01.178298 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7225ce5-4027-4294-9a4f-e0ceee12bab0 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:23:01.180342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-a7225ce5-4027-4294-9a4f-e0ceee12bab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:01.193997 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7225ce5-4027-4294-9a4f-e0ceee12bab0 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:23:01.250188 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083f3d9-6d59-4fe0-a47a-5bd4a22efc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:23:01.250814 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083f3d9-6d59-4fe0-a47a-5bd4a22efc6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:01.251096 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083f3d9-6d59-4fe0-a47a-5bd4a22efc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:01.251373 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083f3d9-6d59-4fe0-a47a-5bd4a22efc6b admin admin] PATH_INFO: `/v3/users/508e8e3ccb30432d9ac0e696dc9a92c6` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:01.252526 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6083f3d9-6d59-4fe0-a47a-5bd4a22efc6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=508e8e3ccb30432d9ac0e696dc9a92c6)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:23:01.254442 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6083f3d9-6d59-4fe0-a47a-5bd4a22efc6b admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:23:01.302259 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6083f3d9-6d59-4fe0-a47a-5bd4a22efc6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '508e8e3ccb30432d9ac0e696dc9a92c6'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:23:01.302903 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6083f3d9-6d59-4fe0-a47a-5bd4a22efc6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '508e8e3ccb30432d9ac0e696dc9a92c6'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:23:01.311524 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6083f3d9-6d59-4fe0-a47a-5bd4a22efc6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '508e8e3ccb30432d9ac0e696dc9a92c6'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:23:01.318456 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6083f3d9-6d59-4fe0-a47a-5bd4a22efc6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '508e8e3ccb30432d9ac0e696dc9a92c6'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:23:01.319069 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-6083f3d9-6d59-4fe0-a47a-5bd4a22efc6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '508e8e3ccb30432d9ac0e696dc9a92c6'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:23:01.336488 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1467/2931] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:23:01 2022] DELETE /identity/v3/users/508e8e3ccb30432d9ac0e696dc9a92c6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:23:01.508361 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80c12c-315c-48a4-a358-678e623c9f13 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:23:01.510021 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-0b80c12c-315c-48a4-a358-678e623c9f13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:01.523763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80c12c-315c-48a4-a358-678e623c9f13 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:23:01.581125 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848395f-663c-4a0b-8a93-9c61b643bef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:23:01.581759 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848395f-663c-4a0b-8a93-9c61b643bef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:01.582041 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848395f-663c-4a0b-8a93-9c61b643bef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:01.582313 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848395f-663c-4a0b-8a93-9c61b643bef9 admin admin] PATH_INFO: `/v3/projects/e9da155613f948a5a168179f7edc64dc` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:01.583420 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848395f-663c-4a0b-8a93-9c61b643bef9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9da155613f948a5a168179f7edc64dc)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:23:01.585358 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848395f-663c-4a0b-8a93-9c61b643bef9 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:23:01.631886 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2848395f-663c-4a0b-8a93-9c61b643bef9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9da155613f948a5a168179f7edc64dc', 'request_id': 'req-2848395f-663c-4a0b-8a93-9c61b643bef9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:23:01.639501 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2848395f-663c-4a0b-8a93-9c61b643bef9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9da155613f948a5a168179f7edc64dc', 'request_id': 'req-2848395f-663c-4a0b-8a93-9c61b643bef9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:23:01.645319 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2848395f-663c-4a0b-8a93-9c61b643bef9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9da155613f948a5a168179f7edc64dc', 'request_id': 'req-2848395f-663c-4a0b-8a93-9c61b643bef9'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:23:01.656689 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2848395f-663c-4a0b-8a93-9c61b643bef9 admin admin] The token cache is being invalidate because project e9da155613f948a5a168179f7edc64dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:23:01.656976 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2848395f-663c-4a0b-8a93-9c61b643bef9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:23:01.657610 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-2848395f-663c-4a0b-8a93-9c61b643bef9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:23:01.659423 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1465/2932] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:23:01 2022] DELETE /identity/v3/projects/e9da155613f948a5a168179f7edc64dc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:23:04.426035 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083f3d9-6d59-4fe0-a47a-5bd4a22efc6b admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:23:04.427443 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-6083f3d9-6d59-4fe0-a47a-5bd4a22efc6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:04.453647 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:04.497656 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083f3d9-6d59-4fe0-a47a-5bd4a22efc6b admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:23:04.556648 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1bfd47-b5c5-4c59-935a-ef1e697ee8a7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18798838de284c84b5b838b3530a2c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:23:04.557285 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1bfd47-b5c5-4c59-935a-ef1e697ee8a7 service placement] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:04.557562 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1bfd47-b5c5-4c59-935a-ef1e697ee8a7 service placement] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:04.557831 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1bfd47-b5c5-4c59-935a-ef1e697ee8a7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:04.559544 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-3c1bfd47-b5c5-4c59-935a-ef1e697ee8a7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:04.570892 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1bfd47-b5c5-4c59-935a-ef1e697ee8a7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:23:04.572513 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1bfd47-b5c5-4c59-935a-ef1e697ee8a7 service placement] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:23:04.582221 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1468/2933] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:23:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:23:11.653696 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=71,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:14.577978 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=48 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:21.675129 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848395f-663c-4a0b-8a93-9c61b643bef9 admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:23:21.676617 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-2848395f-663c-4a0b-8a93-9c61b643bef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:21.756241 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848395f-663c-4a0b-8a93-9c61b643bef9 admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:23:21.760514 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363e5f6-229c-48c4-9e12-fdb17a978665 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9373aa297439487691fae325c0245bb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:23:21.761113 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363e5f6-229c-48c4-9e12-fdb17a978665 service cinder] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:21.761390 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363e5f6-229c-48c4-9e12-fdb17a978665 service cinder] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:21.761660 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363e5f6-229c-48c4-9e12-fdb17a978665 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:21.763625 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1363e5f6-229c-48c4-9e12-fdb17a978665 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:21.821328 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363e5f6-229c-48c4-9e12-fdb17a978665 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:23:21.822763 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363e5f6-229c-48c4-9e12-fdb17a978665 service cinder] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:23:21.831806 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1466/2934] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:23:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:23:31.801047 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52af2e5-0d7e-4e67-b041-26d231e3f317 None None] REQUEST_METHOD: `POST` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:31.801611 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52af2e5-0d7e-4e67-b041-26d231e3f317 None None] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:31.801611 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52af2e5-0d7e-4e67-b041-26d231e3f317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:31.820330 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.auth.core [None req-f52af2e5-0d7e-4e67-b041-26d231e3f317 None None] MFA Rules not processed for user `b1494511685b47519f4aa0674009fbfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81418) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:23:31.821532 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f52af2e5-0d7e-4e67-b041-26d231e3f317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:31.827629 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=57 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:31.828880 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1469/2935] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 01:23:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:23:32.591374 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363e5f6-229c-48c4-9e12-fdb17a978665 service cinder] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:23:32.592961 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-1363e5f6-229c-48c4-9e12-fdb17a978665 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:32.608532 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363e5f6-229c-48c4-9e12-fdb17a978665 service cinder] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:23:32.613420 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa9e6d-bc66-4349-a4f1-4eae27dc14a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18798838de284c84b5b838b3530a2c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:23:32.614027 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa9e6d-bc66-4349-a4f1-4eae27dc14a2 service placement] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:32.614303 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa9e6d-bc66-4349-a4f1-4eae27dc14a2 service placement] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:32.614572 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa9e6d-bc66-4349-a4f1-4eae27dc14a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:32.616281 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-43fa9e6d-bc66-4349-a4f1-4eae27dc14a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:32.629117 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa9e6d-bc66-4349-a4f1-4eae27dc14a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:23:32.630677 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa9e6d-bc66-4349-a4f1-4eae27dc14a2 service placement] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:23:32.641755 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1467/2936] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:23:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:23:41.814950 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=3 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:42.636837 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=6 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:45.432310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52af2e5-0d7e-4e67-b041-26d231e3f317 None None] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:23:45.433914 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-f52af2e5-0d7e-4e67-b041-26d231e3f317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:45.522600 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52af2e5-0d7e-4e67-b041-26d231e3f317 None None] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:23:45.528362 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55223525-ce6d-47ef-8101-330e21e14486 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:23:45.528961 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55223525-ce6d-47ef-8101-330e21e14486 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:45.529239 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55223525-ce6d-47ef-8101-330e21e14486 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:45.529512 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55223525-ce6d-47ef-8101-330e21e14486 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:45.531215 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-55223525-ce6d-47ef-8101-330e21e14486 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:45.542917 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55223525-ce6d-47ef-8101-330e21e14486 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:23:45.544528 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55223525-ce6d-47ef-8101-330e21e14486 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:23:45.554754 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1470/2937] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:23:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:23:48.364923 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa9e6d-bc66-4349-a4f1-4eae27dc14a2 service placement] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:23:48.366400 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-43fa9e6d-bc66-4349-a4f1-4eae27dc14a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:48.446267 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa9e6d-bc66-4349-a4f1-4eae27dc14a2 service placement] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:23:48.450125 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e1812-542c-4516-88cf-a9fdd1e76a0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:23:48.450717 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e1812-542c-4516-88cf-a9fdd1e76a0c service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:48.451000 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e1812-542c-4516-88cf-a9fdd1e76a0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:48.451275 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e1812-542c-4516-88cf-a9fdd1e76a0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:48.453036 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-f76e1812-542c-4516-88cf-a9fdd1e76a0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:48.517331 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e1812-542c-4516-88cf-a9fdd1e76a0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:23:48.518773 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e1812-542c-4516-88cf-a9fdd1e76a0c service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:23:48.529573 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1468/2938] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:23:48 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:23:55.550354 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=37 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:58.523874 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=59 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:25:39.720261 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55223525-ce6d-47ef-8101-330e21e14486 service nova] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:25:39.721718 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-55223525-ce6d-47ef-8101-330e21e14486 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:39.762450 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55223525-ce6d-47ef-8101-330e21e14486 service nova] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:25:39.768604 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2b0fe-720e-4d2d-a81d-e301866ea9f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e0fe2e84f6748c992c9beedf2f5d49b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:25:39.770394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2b0fe-720e-4d2d-a81d-e301866ea9f8 service nova] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:39.770394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2b0fe-720e-4d2d-a81d-e301866ea9f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:39.770394 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2b0fe-720e-4d2d-a81d-e301866ea9f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:39.771811 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e5f2b0fe-720e-4d2d-a81d-e301866ea9f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:39.786164 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2b0fe-720e-4d2d-a81d-e301866ea9f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:25:39.788108 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2b0fe-720e-4d2d-a81d-e301866ea9f8 service nova] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:25:39.800310 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1471/2939] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:25:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:25:49.795330 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=15 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:25:53.024497 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e1812-542c-4516-88cf-a9fdd1e76a0c service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:25:53.033661 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e1812-542c-4516-88cf-a9fdd1e76a0c service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:25:53.037231 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a9211-d2df-432c-9025-c3592ce26940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d64f9bca7c546e4ba7f54c1c47d665e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:25:53.037849 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a9211-d2df-432c-9025-c3592ce26940 service neutron] REQUEST_METHOD: `GET` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:53.038124 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a9211-d2df-432c-9025-c3592ce26940 service neutron] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:53.038393 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a9211-d2df-432c-9025-c3592ce26940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:53.040108 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-377a9211-d2df-432c-9025-c3592ce26940 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:53.102801 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377a9211-d2df-432c-9025-c3592ce26940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:25:53.104257 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377a9211-d2df-432c-9025-c3592ce26940 service neutron] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:25:53.114049 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1469/2940] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:25:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:25:59.441538 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2b0fe-720e-4d2d-a81d-e301866ea9f8 service nova] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:25:59.442975 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-e5f2b0fe-720e-4d2d-a81d-e301866ea9f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:59.459770 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2b0fe-720e-4d2d-a81d-e301866ea9f8 service nova] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:25:59.465651 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f01d5f-3ecb-498d-94c4-4c6aa5aace45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:25:59.466280 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f01d5f-3ecb-498d-94c4-4c6aa5aace45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:59.466564 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f01d5f-3ecb-498d-94c4-4c6aa5aace45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:59.466844 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f01d5f-3ecb-498d-94c4-4c6aa5aace45 admin admin] PATH_INFO: `/v3/users/b1494511685b47519f4aa0674009fbfc` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:59.468087 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f01d5f-3ecb-498d-94c4-4c6aa5aace45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1494511685b47519f4aa0674009fbfc)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:25:59.470034 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f01d5f-3ecb-498d-94c4-4c6aa5aace45 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:25:59.517245 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-46f01d5f-3ecb-498d-94c4-4c6aa5aace45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1494511685b47519f4aa0674009fbfc'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:25:59.517904 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-46f01d5f-3ecb-498d-94c4-4c6aa5aace45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1494511685b47519f4aa0674009fbfc'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:25:59.525253 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-46f01d5f-3ecb-498d-94c4-4c6aa5aace45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1494511685b47519f4aa0674009fbfc'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:25:59.532707 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-46f01d5f-3ecb-498d-94c4-4c6aa5aace45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1494511685b47519f4aa0674009fbfc'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:25:59.533342 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-46f01d5f-3ecb-498d-94c4-4c6aa5aace45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1494511685b47519f4aa0674009fbfc'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:25:59.550919 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1472/2941] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:25:59 2022] DELETE /identity/v3/users/b1494511685b47519f4aa0674009fbfc => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:25:59.720885 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a9211-d2df-432c-9025-c3592ce26940 service neutron] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:25:59.731646 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a9211-d2df-432c-9025-c3592ce26940 service neutron] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:25:59.787977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd3f8c-fada-42b6-b47c-2c95b6f8e53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:25:59.788532 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd3f8c-fada-42b6-b47c-2c95b6f8e53f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:59.788813 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd3f8c-fada-42b6-b47c-2c95b6f8e53f admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:59.789089 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd3f8c-fada-42b6-b47c-2c95b6f8e53f admin admin] PATH_INFO: `/v3/projects/6c0d5a6539764b53852b5741347734a8` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:59.790201 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd3f8c-fada-42b6-b47c-2c95b6f8e53f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c0d5a6539764b53852b5741347734a8)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:25:59.792153 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd3f8c-fada-42b6-b47c-2c95b6f8e53f admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:25:59.837567 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dbfd3f8c-fada-42b6-b47c-2c95b6f8e53f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c0d5a6539764b53852b5741347734a8', 'request_id': 'req-dbfd3f8c-fada-42b6-b47c-2c95b6f8e53f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:25:59.845784 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dbfd3f8c-fada-42b6-b47c-2c95b6f8e53f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c0d5a6539764b53852b5741347734a8', 'request_id': 'req-dbfd3f8c-fada-42b6-b47c-2c95b6f8e53f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:25:59.850991 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dbfd3f8c-fada-42b6-b47c-2c95b6f8e53f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c0d5a6539764b53852b5741347734a8', 'request_id': 'req-dbfd3f8c-fada-42b6-b47c-2c95b6f8e53f'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:25:59.862755 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dbfd3f8c-fada-42b6-b47c-2c95b6f8e53f admin admin] The token cache is being invalidate because project 6c0d5a6539764b53852b5741347734a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:25:59.863040 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dbfd3f8c-fada-42b6-b47c-2c95b6f8e53f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:25:59.863711 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-dbfd3f8c-fada-42b6-b47c-2c95b6f8e53f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:25:59.865253 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1470/2942] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:25:59 2022] DELETE /identity/v3/projects/6c0d5a6539764b53852b5741347734a8 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:25:59.869191 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f01d5f-3ecb-498d-94c4-4c6aa5aace45 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:25:59.870577 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-46f01d5f-3ecb-498d-94c4-4c6aa5aace45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:59.883354 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f01d5f-3ecb-498d-94c4-4c6aa5aace45 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:25:59.940400 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683cd75b-7cef-49e5-ae85-0c61cf4f4433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:25:59.941030 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683cd75b-7cef-49e5-ae85-0c61cf4f4433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:59.941311 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683cd75b-7cef-49e5-ae85-0c61cf4f4433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:59.941583 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683cd75b-7cef-49e5-ae85-0c61cf4f4433 admin admin] PATH_INFO: `/v3/users/ec177049ed9c43efa589090bf9d42914` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:59.942710 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683cd75b-7cef-49e5-ae85-0c61cf4f4433 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec177049ed9c43efa589090bf9d42914)` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:25:59.944681 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683cd75b-7cef-49e5-ae85-0c61cf4f4433 admin admin] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:25:59.990634 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-683cd75b-7cef-49e5-ae85-0c61cf4f4433 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec177049ed9c43efa589090bf9d42914'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:25:59.991271 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-683cd75b-7cef-49e5-ae85-0c61cf4f4433 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec177049ed9c43efa589090bf9d42914'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:25:59.999289 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-683cd75b-7cef-49e5-ae85-0c61cf4f4433 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec177049ed9c43efa589090bf9d42914'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:26:00.006283 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-683cd75b-7cef-49e5-ae85-0c61cf4f4433 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec177049ed9c43efa589090bf9d42914'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:26:00.006906 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.notifications [None req-683cd75b-7cef-49e5-ae85-0c61cf4f4433 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec177049ed9c43efa589090bf9d42914'} {{(pid=81418) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:26:00.015209 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:26:00.027573 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1473/2943] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 01:25:59 2022] DELETE /identity/v3/users/ec177049ed9c43efa589090bf9d42914 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:26:00.201061 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd3f8c-fada-42b6-b47c-2c95b6f8e53f admin admin] Authenticating user token {{(pid=81417) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:26:00.202581 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.fernet_utils [None req-dbfd3f8c-fada-42b6-b47c-2c95b6f8e53f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81417) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:26:00.216008 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd3f8c-fada-42b6-b47c-2c95b6f8e53f admin admin] Validating token access rules against request {{(pid=81417) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:26:00.242167 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:26:00.283951 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a6934b-ba0b-47a6-b4c1-20588ad56ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dea2911925e405b852c2fe0f9bee81d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '724534edd35a41c585d8ed624130e96a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81417) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:26:00.284589 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a6934b-ba0b-47a6-b4c1-20588ad56ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:26:00.284871 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a6934b-ba0b-47a6-b4c1-20588ad56ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:26:00.285146 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a6934b-ba0b-47a6-b4c1-20588ad56ee3 admin admin] PATH_INFO: `/v3/projects/849e60970e2342b08ce58582752ef719` {{(pid=81417) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:26:00.286254 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a6934b-ba0b-47a6-b4c1-20588ad56ee3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=849e60970e2342b08ce58582752ef719)` {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:26:00.288321 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a6934b-ba0b-47a6-b4c1-20588ad56ee3 admin admin] RBAC: Authorization granted {{(pid=81417) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:26:00.334977 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-77a6934b-ba0b-47a6-b4c1-20588ad56ee3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '849e60970e2342b08ce58582752ef719', 'request_id': 'req-77a6934b-ba0b-47a6-b4c1-20588ad56ee3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:26:00.341734 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-77a6934b-ba0b-47a6-b4c1-20588ad56ee3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '849e60970e2342b08ce58582752ef719', 'request_id': 'req-77a6934b-ba0b-47a6-b4c1-20588ad56ee3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:26:00.347426 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-77a6934b-ba0b-47a6-b4c1-20588ad56ee3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '849e60970e2342b08ce58582752ef719', 'request_id': 'req-77a6934b-ba0b-47a6-b4c1-20588ad56ee3'} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:26:00.358562 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-77a6934b-ba0b-47a6-b4c1-20588ad56ee3 admin admin] The token cache is being invalidate because project 849e60970e2342b08ce58582752ef719 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81417) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 01:26:00.358850 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-77a6934b-ba0b-47a6-b4c1-20588ad56ee3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:26:00.359485 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG keystone.notifications [None req-77a6934b-ba0b-47a6-b4c1-20588ad56ee3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81417) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 01:26:00.360982 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: [pid: 81417|app: 0|req: 1471/2944] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 01:26:00 2022] DELETE /identity/v3/projects/849e60970e2342b08ce58582752ef719 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:26:10.025345 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:26:10.355540 dv-cinder-857528-1-iscsi devstack@keystone.service[81417]: DEBUG dbcounter [-] [81417] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81417) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:28:10.403730 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683cd75b-7cef-49e5-ae85-0c61cf4f4433 admin admin] Authenticating user token {{(pid=81418) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:28:10.405144 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-683cd75b-7cef-49e5-ae85-0c61cf4f4433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:28:10.480560 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683cd75b-7cef-49e5-ae85-0c61cf4f4433 admin admin] Validating token access rules against request {{(pid=81418) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:28:10.539190 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7126fad-6e7d-4a46-b1a7-73b3d81672fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18798838de284c84b5b838b3530a2c43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fda69e8c2d5e466dbc1244b2f76296a3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81418) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:28:10.539818 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7126fad-6e7d-4a46-b1a7-73b3d81672fd service placement] REQUEST_METHOD: `GET` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:28:10.540096 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7126fad-6e7d-4a46-b1a7-73b3d81672fd service placement] SCRIPT_NAME: `/identity` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:28:10.540367 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7126fad-6e7d-4a46-b1a7-73b3d81672fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81418) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:28:10.542059 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.fernet_utils [None req-d7126fad-6e7d-4a46-b1a7-73b3d81672fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81418) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:28:10.553416 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7126fad-6e7d-4a46-b1a7-73b3d81672fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:28:10.554981 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7126fad-6e7d-4a46-b1a7-73b3d81672fd service placement] RBAC: Authorization granted {{(pid=81418) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:28:10.564869 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: [pid: 81418|app: 0|req: 1474/2945] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 01:28:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:28:20.560620 dv-cinder-857528-1-iscsi devstack@keystone.service[81418]: DEBUG dbcounter [-] [81418] Writing DB stats keystone:SELECT=58 {{(pid=81418) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}